Two tracked files had been edited directly in the work-PC relay clone, which
only fetches bundles and pushes them to GitHub. Nothing there reaches gitea, and
the next bundle pull would have overwritten both silently. Brought across here,
with three changes to what was written.
Kiosk links now need the route AND the plugin. The edit swapped hasRoute for
isPluginEnabled, which trades one wrong answer for another: a full build stages
every route, so a site that turns printedparts or usb OFF still saw a link to a
kiosk its backend refuses - the symptom being fixed - but isPluginEnabled fails
OPEN when it cannot reach the plugin list, and a copied registry can report a
plugin that was never staged into this build. That is what the comment above
those lines was already warning about. Both tests together answer both states.
The wizard shows the filename it is uploading. The browser puts it in chrome the
full-screen wizard hides, so an operator picking a file got no confirmation it
was the one being sent.
Image previews get a cache-buster. The stored path does not change between
uploads, so the preview showed the PREVIOUS logo or blueprint and the upload
read as having silently failed. Display only - saveStep persists form values,
never these, so no cache-busted URL is ever written to a setting.
The wizard also loads the current site logo on mount, so re-running it does not
present an empty slot as though branding had never been set. The shipped-asset
fallback stays a root-absolute path: the template applies withBase, and prefixing
it here as well produced /shopdb/shopdb/ge-aerospace-logo.svg on a subpath mount.
Frontend 391 passing.
Scope: product
A local 'dotnet build' leaves agent/*/obj and agent/*/bin full of absolute
host paths. They are gitignored, but the export rsyncs the WORKING TREE, so
.gitignore does not apply and the scrub gate stopped the whole export - which
is what it is for, but it made publication depend on whoever built the agent
last having cleaned up.
Same shape as the stray .env.bak the excludes already document. Note the trap
that follows from it: an --exclude also PROTECTS a path from --delete, so the
copies the first failed run had already placed in the publication tree had to
be removed by hand before the next run could pass.
Scope: product
Three defects that all shared a shape: the restore path looked checked and was
not.
An unreadable data revision was treated as a legacy set. backup.sh discards the
mysql error when reading alembic_version, so a set could be written with an
empty datarev; the compatibility check then fell through to the code-to-code
head comparison - the very check datarev was introduced to replace - and badged
the set restorable on no evidence at all. A manifest that CARRIES the key and
left it empty is now refused, while one with no key at all is still the genuine
legacy case and keeps the fallback. backup.sh records the failure in run history
rather than passing it over in a log line, since the set it just wrote cannot be
restored.
Dumps were bound to the schema they came from. mysqldump --databases bakes
CREATE DATABASE and USE <name> into the file, so restoring onto a site whose
MYSQL_DATABASE differs loaded every row into the dump's schema, left the live
database untouched, and reported success. New sets are dumped without it and
load into the schema named at restore time. Existing sets are still bound, so
the gate reads which schema they name and refuses a mismatch instead of
restoring beside the live data. Also dropped --no-tablespaces into the main
dump: it needs the PROCESS privilege the app user does not have on MySQL 8.
The gate's own test carried a copy of the gate. test-restore-datarev.sh
re-implemented the predicate in its own shell functions, so it would have stayed
green through any drift in restore.sh - including the plugin-chain hole fixed in
the previous commit, which it could never have caught. restore.sh now takes
RESTORE_CHECK_ONLY, which stops it immediately after the gates having touched
nothing, plus overridable paths for the migration and plugin trees. The test
drives the shipped script and asserts on its exit code, so the verdict under
test IS the verdict that ships. Confirmed non-vacuous by planting a violation:
disabling the plugin revision check turns the suite red.
Nine cases now, up from five, covering both plugin-chain directions, a plugin
the build does not have, and the schema binding.
Moving the schema check earlier was prompted by that test: it had been placed
after the safety copy, so it needed a live database to reach a verdict it can
reach for free. It sits with the other gates now and stops before anything is
written.
Round-tripped against a real stack after the dump format change: backup, mutate,
restore, data back to backup-time state, and the new dumps confirmed to carry no
schema name.
Suite 2607 passed / 9 skipped.
Scope: product
The compatibility gate read alembic_version and nothing else, but plugin schemas
move independently of core. Roll the code back to an older plugin revision and
the core head can be identical, so a dump carrying plugin tables the code has
never heard of passed the gate, loaded - destroying what was there - and only
then failed in 'flask plugin upgrade-all' with "Can't locate revision". A failed
restore that has already replaced what it was restoring over is the worst
outcome this feature can produce, and it was reachable from an ordinary rollback.
Backups now stamp the applied revision of every alembic_version_<plugin> table
alongside the core one, read from the dumped database the same way. restore.sh
checks each against that plugin's own versions directory and refuses before
touching anything, naming the plugin and the revision to deploy past. A plugin
in the dump that this build does not have at all is refused for the same reason.
The list endpoint's compatibility badge asks the same question, so the UI cannot
offer a set the restore will reject.
Sets taken before the stamp existed carry no dataplugins and keep their old
core-only verdict. Their plugin revisions are unknowable, and making every
existing set unrestorable to close this would be a worse trade than the gap.
Proven against a real stack, reproducing the exact case: core head identical,
endpoints data stamped at a revision the build does not have. The restore
refused, the live row was still there afterwards, and it stopped early enough
that the pre-restore safety copy was never needed. The same set with its true
revision restored normally.
Suite 2605 passed / 9 skipped, with the gate covered both ways plus the legacy
fallback.
Scope: product
Closes the three defects in the imported restore path, at Ben's request. Each
was proven against a real stack before and after: two backup sets with different
data and different instance files, then a restore of the OLDER one, which is the
case that was broken.
A restore no longer depends on a tab staying open. The backup sidecar has no
Flask, so it loads the dump and leaves a sentinel for the api container to run
the migrations. The only thing that ever acted on that sentinel was the poll
loop of the modal that queued the restore, so closing the tab - or letting a
token expire - during the minutes a load takes left the data at the old revision
under new code, every query on a newer column failing, and nothing retrying. A
watcher in the api container now claims and runs it. The endpoint no longer runs
the upgrade at all; it only makes sure something is pending, which is also how a
failed upgrade is retried. That takes Alembic out of the request path, where it
could outrun the gunicorn worker timeout and be killed mid-chain, leaving a
half-applied revision the next attempt trips over.
The claim is a rename, because rename(2) is atomic: several workers and a
browser poll can race the same sentinel and exactly one wins. A failed upgrade
parks itself as .upgrade-failed rather than re-arming, since putting it back
would rerun the same broken chain on every poll forever.
Instance files come from the set being restored. The restic call asked for
'latest' regardless of which set was named, so restoring anything but the newest
paired that set's DATABASE with the NEWEST snapshot's uploads and plugins.json -
a mismatch nothing downstream could detect. Now scoped by the set's own tag,
which backup.sh has always written. The tree is replaced rather than restored
over, so files created after the snapshot are gone afterwards: a restore has to
put the site back to the snapshot, not merge into it. Done as a staged swap
rather than 'restic restore --delete --target /', because that aims a delete
pass at the container root and the cost of getting its filter wrong is the whole
filesystem.
Nothing destructive happens before a copy is taken. The dump carries DROP TABLE
per table with no transaction around it, and the instance tree is replaced
wholesale, so a wrong set or a failure part-way through left nothing to go back
to. restore.sh now writes a pre-restore-<stamp>/ set - dump, instance tarball
and a README saying what it is and how to load it - and refuses to touch
anything if that copy cannot be written.
Two more found while proving the above, both of which stopped the overlay
working at all:
* Every script under scripts/backup/ was committed non-executable, while every
other .sh in the repo is 755. The compose entrypoints are
/opt/backup/watch.sh and backup.sh from a read-only bind mount, so the backup
and watcher containers could not start from a clean checkout on any machine,
and nothing could fix it at runtime.
* Dockerfile.backup could not build without the corporate CA staged. A plain
COPY of the gitignored cert fails outright; the trailing-glob form does too,
because BuildKit fails a COPY whose pattern matches nothing - which is exactly
the case being handled. Copying the directory and testing for the file inside
it is the form that works both ways. Verified by building the image with no
cert present.
Suite 2600 passed / 9 skipped. The finalizer has unit tests covering the atomic
claim, the parked failure and the endpoint no longer running the upgrade.
Scope: product
Six findings from a four-agent review of the imported work, plus the contract
bump the import owed. Nothing here changes what the imported features do.
The contract version was not bumped for a change that widens what a plugin may
declare. The asset-panel vocabulary gained two tab groups (serviceprocess,
reports), a datetime column format, a map.body key and a tabs section that
renders as a table. 0.29.0 bumped for exactly this class of change when the
optional group was added, so this is 0.31.0 with a ledger row. All four are
optional and resolve server-side when absent, so a plugin declaring none of them
is unaffected.
Site Backups now answers honestly where it has nothing behind it. The overlay is
a compose profile: a separate container writes sets onto a shared volume and the
api container only reads them. A Windows/IIS install runs no such container and
the default root resolves to a drive-letter path nothing creates, so Back Up Now
either failed on PermissionError or reported a run queued that would never
happen, under a banner reading "No verified backup yet" - which describes a
backup that failed, not a feature that was never installed. The three write
routes refuse with the same 404 shape the configuration-policy routes use for a
disabled feature, the list and status routes report whether a store exists, and
the page says so instead of offering buttons.
Two listAll callers still read a response that is not there. listAll resolves to
a plain array through fetchAllPages, so response.data.data throws: PC Types
settings rendered empty, and in Configuration Policies the throw happened inside
the shared try, so PC types, applications, the action catalog and monitors were
all left unassigned by one bad read.
The agent package upload cannot succeed on IIS as shipped. MAX_CONTENT_LENGTH
went to 256 MB to admit the ~69 MB tray binary, but IIS request filtering caps a
body at 30,000,000 bytes and answers 413 before waitress sees the request, so
web.config needs the matching requestLimits.
The backup image could not be built by anyone outside a TLS-intercepting
network. Dockerfile.backup did a plain COPY of a gitignored corporate root CA,
and a COPY of a missing file fails the build. The trailing glob makes it a no-op
when the cert is absent.
endpointstates.domain was the one new string column written without a clamp.
Under STRICT_TRANS_TABLES an over-long value raises Data too long and aborts the
whole check-in, so a single malformed report would stop a machine reporting
telemetry and events entirely.
Not fixed here, and left with upstream: the restore path finalizes its schema
upgrade only from a browser poll loop, so closing the tab mid-restore leaves the
data behind the code; it restores instance files from the newest snapshot rather
than the set that was asked for; and it takes no safety dump before the
destructive load. All three are in the Docker-only overlay, which this site does
not run, and none can be validated here.
Scope: product
Ports 37 of Benjamin Baker's commits from the wilm branch, mirrored from the
corporate GitHub at 7905917c. wilm rides the publication lineage and shares no
ancestor with wj, so this is a filtered tree patch rather than a merge:
94 files, and every path the publication export strips (docs/, tools/, mcp/,
CLAUDE.md, start-*.sh, .gitea/, the three excluded tests) is held back, because
a raw diff would delete each of them from this tree.
What comes across:
* Endpoint telemetry, phase 2. Physical disk health from two sources, NIC link
state, boot type, Defender health, processor/memory/chassis inventory, the
listening TCP set, mapped network drives and Windows Update history. Five
additive migrations, 0012 through 0016, off the 0011 head.
* Mapped drives are collected by a probe running in the interactive user's
session, because they do not exist in the agent's session 0. Two Windows
sources can disagree about one drive and the worse status wins, so a
half-broken share is never rendered as healthy.
* Windows events are reported once each, keyed on the channel's own record id,
instead of being re-sent whenever a report overlapped the last one.
* PC Detail rework: Details cards flow, the hand-written Network card gives way
to a full address history, Storage and disk health merge into one card with
two sections, Operator Reports and Corporate Tools get their own tabs.
* An opt-in restic backup overlay with a third container, restore gated on the
dump's own data revision rather than the code head, and backup/restore history.
* The agent's version is read from the binary's PE version rather than trusted
from the csproj, and the Fleet view gains an Agent Version column.
* The enqueue route now asks caller_holds instead of the owner's permissions,
which closes the scoped-token elevation on run_script, update_app and
system_restart.
Held back deliberately:
* agent/installer/install-shopdbagent.ps1 stays deleted, as upstream deleted it.
The installer that reaches a bay is generated by _install_script() in
plugins/endpoints/api/siteagent.py; the standalone script had stopped being
shipped and was collecting fixes no endpoint ever ran.
* The .gitignore hunk adding docs/*.md. Correct on the publication lineage,
where docs/ becomes the wiki and its absence is how the gates detect a
published checkout. Here it would silently ignore every new doc.
* Their deletions of docs/DEV-BUILD-TEST.md and docs/PLUGIN-HOOKS.md. Upstream
those were stubs; here PLUGIN-HOOKS.md is the 31KB document CLAUDE.md points
readers at.
* Five zero-byte files committed by an accidental `git add -A`, a run log from
the author's own machine, and docs/notes/, which is working scratch that says
so in its own header and belongs in devnotes/ on this side.
Carried with the import so it lands green: the 17 new routes are described in
docs/api-inventory.json with the spec and project map regenerated, and two new
frontend specs read fixtures through fileURLToPath(URL) which node rejects under
jsdom's own URL global.
Suite 2591 passed / 9 skipped; frontend 391 passed; naming, style and doc gates
clean.
Scope: product
Phase 3 of the device-identity proposal moves device identity to
HKLM:\SOFTWARE\GE\ShopDB\DeviceId, which survives the profile cleanup and disk
hygiene that eats C:\Enrollment and needs no new key: the collector already
opens that path for BaseUrl and ApiToken. This is the backfill that gets a
value in there without re-imaging anything.
The source depends on the pc-type, and that is the whole point. DeviceId names
the DEVICE, while a machine number answers "which bay is this" - a different
fact that coincides for exactly one family:
CMM the machine number IS the instrument id (CMM4), which is why
cmmid.txt has always doubled as identity. Copied.
part marker the machine number is the OPERATION, and several markers serve
one operation. Copying it would name the operation as the
device, which the collector refuses as the wrong kind of asset
because a machine filed under a device label reads as a marker
everywhere downstream. Never copied.
Keyence, no per-bay id has ever existed; asset-id.txt is the only
Genspect, identity these have.
wax-trace
anything else not a device bay, so nothing is written. DeviceId is not
consumed for those pc-types and a value left there is a
landmine for the day one gains a device spec.
Resolution per family mirrors Report-AssetToShopDB.ps1, including the 9999
placeholder skip and reading WOW6432Node before the native eDNC key.
Refuses to overwrite a DeviceId that disagrees, unless -Force. asset-id.txt is
written once at imaging and nothing updates it on a device swap, so a stale
file clobbering a corrected registry value would reintroduce the exact defect
this move exists to fix. GE-Enforce already reached the same conclusion for
machine-number.txt.
Verified in the win11 VM across ten cases: not-a-device bay, part marker,
CMM precedence, idempotent re-run, machine-number exclusion, eDNC fallback,
9999 skip, stale-file refusal (exit 2), -Force, and -WhatIf. The part-marker
guard was then planted - flipping it to treat the machine number as identity
writes operation 0613 into DeviceId, so the passing test is not vacuous.
Scope: product
ADR-005 split machines from measuring tools on the grounds that they are two
asset classes. It named the metrology population as CMMs, vision systems, wax
trace and Genspect; the plugin was then built against a docstring naming
calipers, micrometers and thread gages. Those are not the same population, and
the difference is the whole decision: bay-mounted capital equipment with a
controlling PC on one side, hand tools in a drawer on the other.
Sixteen months on, measuringtools holds zero rows against 204 machines. The
metrology inventory waiting to be entered is 134 instruments - 35 CMM, 32 form
tracer, 23 vision, 18 wax, 14 measuring, 8 Genspect, 2 eddy, 1 spline - every
one bay-mounted with a controlling PC, and not one hand gage among them. CMM is
already a machinetype, and Part Marker is already a machine while Form Tracer
is a measuring tool, so the line is already drawn somewhere no operator can
predict.
Retires the plugin and the measuring_tool asset type. Calibration is not lost:
it moves onto machines as four nullable columns with status still derived at
read, which is a gain rather than a port because a machine could not record
calibration at all before. ADR-005's one concrete objection - that "show me all
measuring tools" would need an enumeration of type names - is answered with
machinetypes.ismetrology, so it stays a single-column filter.
Taken now because measuringtools is empty. After the 134 rows are entered they
carry relationships, audit history, map positions and collector links, and the
migration stops being a drop.
Scope: product
The adoption guide told an adopting site that a missing EventSaver.ini makes
the binary fall back to "the reference site's file server". That was true of
EventSaver.cs until the ADR-015 fix set Folder to empty, and EVENTSAVER.md was
updated then while this page was not, so the two documents disagreed about what
a missing ini does.
Current behaviour is that it shows nothing, which is the point: failing visibly
beats displaying another site's slides. Says that, and adds the case the old
text was accidentally describing - a .scr compiled before the fix still carries
the path, so a binary somebody was handed is not the same as one they built.
Scope: product
CONTRIBUTING.md still said "main is the product every site runs and the
publication mirror exports" and framed wj/wilm as site branches feeding it.
That was true when main was the only thing anyone deployed. It is not now:
WJ production runs wj, the /ops instance runs dev, and main deploys nowhere.
Nothing in the repository mentioned dev at all, so the branch a deploy script
refuses to run without was undocumented.
Replaces that section with the branch/instance table and the dev -> wj -> main
promotion order, and says why promoting before validating is the wrong way
round. Keeps the wilm note but moves it up with the reason it cannot be
merged locally: it sits on the publication lineage and shares no ancestor.
CLAUDE.md gains the branch-to-server pointer and records that
export-github.sh now builds a dist only for wj and dev.
docs/DEPLOY.md is deliberately left alone: `git pull origin main` is correct
there. That document is the generic connected-Docker deploy another site
follows, and main is exactly what another site adopts.
Scope: product
Taken from the other site, who hit it in production today (their commit
db75ca7): agents that fail the TLS handshake retry in a loop, and with four
blocking sync workers a handful of them pinned every worker. Real UI requests
queued for 20+ seconds while the CPU sat idle - the box was not busy, it was
blocked.
gthread, 4 workers x 8 threads, and a 30-second timeout rather than 60. A stuck
connection now holds one thread out of 32 instead of one worker out of four, and
it lets go in half the time, which is the difference between absorbing a retry
storm and queueing behind it.
Their server, our agent. The client behaviour that caused it is the endpoint
agent, so it will reach any site running one - which makes this a product fix
rather than something that belongs on one site's branch.
Applied to the Dockerfile as well as docker-compose.yml. A compose file
overrides the image CMD, but docker-compose.airgap.yml, .channel.yml and
.pull.yml do not specify a command at all, so they take the image default - and
so does anybody running the image by hand. Fixing only compose would have left
every air-gapped install on sync workers.
WJ production is unaffected either way: it runs waitress under IIS, already with
--threads=8, so it was never exposed to this. That is worth knowing before
anyone treats it as a prod hardening item that was missing.
Scope: product
The rule for this site, stated plainly: only `wj` reaches /shopdb (WJ
production), only `dev` reaches /ops, and `main` is the curated release other
sites adopt and deploys nowhere. The tooling did not know that, and in two
places actively assumed the opposite.
export-github.sh refused --dist on any branch but main, because production was
assumed to install main. It does not - it installs wj. That guard fired today: a
wj export produced its bundle and then died at the dist step, so the dists on
the share came from the main export that happened to run next. The code going to
production and the frontend going to production would have come from different
branches, silently, and nothing downstream could tell.
Now one branch builds one dist: wj builds the /shopdb dist, dev builds the /ops
dist, and any other branch is refused because a dist it built could only ever be
installed by mistake. Each dist carries a .dist-source stamp naming the branch,
commit, base path and build time.
The two deploy scripts copy from the clone on WHATEVER branch it is sitting on
and only ever checked the destination (the web.config mount path), so a clone
left on the wrong branch deployed the wrong code to a correctly-identified
server without a word. Both now refuse unless the clone is on their branch, and
both read the dist's .dist-source and refuse a frontend built from anywhere
else. A directory of hashed filenames says nothing about where it came from, and
robocopy /MIR will mirror whichever one it is pointed at.
pull-shopdb-bundle.cmd and update-shopdb-github.cmd silently defaulted to main.
That was safe when main was the only thing anyone deployed; with three branches
it is a way to carry the wrong one to a server without ever typing its name. The
argument is required now, and the error lists what each branch is for.
tools/mirror-wilm.sh is the missing half of fetch-wilm-branch.cmd: that script
only puts a bundle on the share, and wilm does not appear on gitea by itself.
The new one fetches the carried bundle and pushes wilm to gitea, printing the
bundle's age first - a mirror that looks authoritative and is a week old is
worse than no mirror - and noting that wilm sits on the publication lineage, so
it shares no ancestor with main/wj/dev and cannot be merged here.
Scope: product
The last two items from the run.
The PC detail page has always rendered a Business Unit row, and the API has
always accepted the field. Only the form had no control, so it could be read and
never set from the UI - a value that appeared the moment somebody set it through
the API and could not be reached otherwise. The form has the picker now, loaded
with listAll() rather than a capped page.
Global search excluded inactive assets, so a retired record existed, was
reachable by URL, and could not be found by anybody who did not already know
where it was - including whoever wanted to restore it, which the retire dialog
promises they can. Retired assets are searchable now.
That is only safe with the ordering, so it is part of the change rather than a
follow-up: somebody searching a machine number almost always wants the machine
still on the floor, so a retired match drops 60 points of relevance, sorts below
every live one, and carries a 'retired' flag the results page renders as a
badge. Indistinguishable retired results would be worse than none.
Four tests, verified by planting the isactive filter back: three fail.
Scope: product
The last two kiosk findings.
Keypad entry collected EVERY digit in an item's code, and the shipped prefix
contains one: '3DP0004' collapsed to 30004, so an operator typing the 4 printed
on the label was told no part matches that barcode and had to work out that the
leading 3 belonged to the prefix. The number on a label is the number at the
end, so the trailing digit run is what is compared now. An ambiguous number is
still refused rather than guessed - taking from the wrong bin is worse than
being asked to scan the label.
A parts kiosk that had never been provisioned drew its normal "Scan the barcode
on the bin" screen and answered "Not found" to every scan, which reads as "we
do not stock that part" rather than "this screen was never set up". The server
cannot say which: the routes answer a bare 404 on purpose, so a caller without
the device token does not learn the door exists, and that stays. But the BROWSER
knows whether it holds a token, and saying so discloses nothing - an
unprovisioned screen now says it is not set up and how to fix it.
For the case the client genuinely cannot distinguish - kiosk switched off versus
a part that is not stocked, both a 404 by design - three misses in a row add a
line suggesting the kiosk may have been switched off. Truthful without the
server having to leak anything.
Three tests, verified by planting the all-digits match back.
Scope: product
Three findings, all about a screen or a log saying something that is not so.
Editing a PC wrote an audit entry for its named fields and nothing for the two
changes that matter most when somebody investigates later: the primary IP
address, which is how the machine is found, and the remote-access protocols and
ports, which is how somebody connects to it. Both are on the same form, in the
same request, and both were invisible afterwards - a PC that moved address had
no history of moving, and a PC that gained a VNC listener had no record of
gaining one. Both are recorded now, and re-sending the same value still records
nothing, because an audit log full of non-events is one nobody reads.
While proving that, a silent discard turned up in the same lines: both the
create and the update path did `if ip_comtype:` around the IP write and did
nothing at all when the 'IP' communication type row was missing. Somebody typed
an address, saved, got a 200, and the address was gone. A missing reference row
is a seeding gap, not a reason to drop what was typed, so it is created on
demand.
The PCs list rendered a 403 as "No computers found", which tells a person
without the permission that the fleet is empty and sends them looking for the
data rather than for the permission. It now says which permission is missing.
The Site Policies and Configuration Policies pages rendered their empty table
and an Add button while the policy feature flag was off - every route answers
404 "Policy feature is not enabled", so the button could never work. Both pages
now say the feature is off and that the empty list is a consequence of that,
not of nothing being configured.
Nine tests. The audit entries verified by planting: remove either and two fail.
Scope: product
You are right that this is the one we keep hitting. perpage is capped at 100
server-side, and the cap is applied SILENTLY: a request for 500 returns the
first 100 rows with a 200, so a picker built that way shows the first 100
options and drops the rest with nothing anywhere reporting a problem. The
missing vendor simply is not in the list.
Twelve call sites were still doing it - vendor and business-unit pickers on the
machine and measuring-tool forms, and the type lists behind PCs, machines,
printers, network devices, measuring tools and configuration policies. Each now
either pages through with listAll()/fetchAllPages() or asks for what it can
actually have. vendorsApi and businessunitsApi gained listAll(), which they
lacked, which is why those two sites reached for 500 in the first place.
The gate is the point. tests/test_core/test_perpage_cap.py reads every .vue and
.js source under frontend/src and plugins/*/frontend and fails on any perpage
over 100. It has to read the SOURCE rather than the running app, because the
response is a valid 200 either way - no runtime assertion can see this one.
Verified by planting a perpage: 500 back into MachineForm: the gate names that
exact line.
Separately, /api/auditlogs built its own meta with the pagination fields flat
and a 'pages' key, while every other list returns
meta.pagination.{page,perpage,total,totalpages,hasnext,hasprev}. A client
written against the documented contract found nothing where it looked, on the
one list whose whole purpose is being read by somebody checking what happened.
It uses paginated_response now.
Also carried here, from the previous batch's working tree: Asset.to_dict emits
ownlocationid/ownmapx/ownmapy/ownlevelid alongside the resolved values, and the
PC form binds to those. The merge of an inherited location is right for a reader
and wrong for a form, which could not tell an owned value from a derived one and
wrote the inherited one back as the record's own.
Scope: product
Reported from the floor: somebody types the employee's whole name into the
notification or recognition picker and presses Enter instead of clicking the
suggestion, and the entry does not pull from the employee database.
Two causes, one symptom.
The search matched the WHOLE query against first name or last name, so "Dana
Reyes" found nobody - no single column holds that string. Typing a full name is
what a person does before pressing Enter, and it was the one thing guaranteed to
return an empty list. A multi-word query is now treated as a full name: every
word must appear somewhere in the name, in either column, so "Reyes Dana" works
too, and "Dana Reyes" does not also return Sam Reyes. A single word still
searches broadly, and an SSO still resolves. Both directory backends changed,
the self-hosted tables and the external HR database.
With no suggestion to click, the form then turned the typed text into
`NAME:<text>` - a label linked to nothing. On screen that chip is identical to a
real selection; the difference only shows up later as no photo, no directory
link, and a recognition the person it is about cannot be found by. Both pickers
(the notifications form and the shared EmployeeSearch component) now look a
typed name up before deciding it is free text, through one helper.
An exact full-name match wins over a longer partial one, because somebody typing
"Dana Reyes" means that person and not "Dana Reyes-Smith". A genuinely ambiguous
name is NOT guessed at - the candidates are shown and the operator picks, which
is exactly when picking wrong matters most. A directory that cannot be reached
reports failure rather than silently producing a label.
Eleven tests. The search rules verified by planting the single-term query back.
One note for whoever reads the spec: the failure-branch test uses a thenable
that rejects rather than a mock returning a rejected promise. The runner
collects such a promise as an unhandled error even when the code under test
catches it, and fails the very test that proves the catch works.
Scope: product
Two operator-visible defects and three doc corrections, closing the tail of the
2026-09-03 run.
THE BADGE. The two kiosks read the same physical badges and disagreed. The
parts kiosk understood both shapes a scanner emits - the bare SSO and the PayNo
wrapper 0<digits>BZ - while the USB kiosk compared the raw string to an SSO, so
the wrapper resolved to nobody. An operator whose keyboard-wedge scanner emits
it could take a 3D-printed part and could not check out a drive, with the same
badge, at two cabinets a few metres apart.
The shape now lives in shopdb/utils/badges.py on the contract surface, because
a plugin may not import another plugin and because a badge is a property of the
site rather than of a cabinet. Both kiosks use it, and both store the DIGITS
rather than the string that arrived: one row per spelling makes a custody
history that cannot be joined to itself.
That closed a subtler hole in the same place. str.isdigit() is true for
Arabic-Indic and other decimal forms, and int() parses them, so a badge written
in those characters was accepted, matched a REAL employee, and was then stored
as an SSO string nothing else matches. ASCII digits only. An empty scan and an
unreadable one now get different answers, because they are different problems
for the person at the cabinet.
THE IMAGES. /api/applications/image/<filename> and /api/models/image/<filename>
required a permission, and a browser fetching <img src="..."> sends no
Authorization header - so the gate did not protect the file, it guaranteed the
picture never appeared. Both are ungated now, like the four sibling image routes
(branding logo, map blueprint, employee photo, part picture), and registered in
PUBLIC_ENDPOINTS with that reasoning.
/api/printers/lookup?fqdn= never matched anything: both arguments were compared
against Communication.ipaddress, so a hostname lookup 404d on printers whose
hostname was on the record. An FQDN now matches the hostname, with a fallback to
the short name.
The slide manager gained the per-slide hold time. The PATCH route and the feed
have always carried `seconds`; nothing called it, so the one documented
per-slide control was curl-only.
Docs: API-REFERENCE listed the employee lookups and the whole notifications
surface as unauthenticated when they answer 401, and called both kiosks
"deliberately open" when they are off by default and refuse anything without the
site's device token. The installer download says applications.download, which is
what the code requires. api-inventory.json said printers/lookup was
jwt-optional.
Fourteen tests. The badge rule verified by planting the old per-kiosk behaviour:
three fail, one in each kiosk.
Scope: product
The validation tail from the 2026-09-03 run. Each is small; together they are
the difference between a field that checks its input and one that looks like it
does.
Subnets accepted any string containing a slash, so 'notanip/xx' was stored -
and _devices_in_subnet then swallowed the ValueError from ip_network and
returned an empty device list, so the subnet looked real and permanently empty.
It is parsed with the same library that has to read it later, a prefix length is
required (ipaddress reads a bare 10.1.1.0 as a /32, which is legal and never
what somebody typing a subnet meant), and the stored form is normalised so
10.1.1.5/24 and 10.1.1.0/24 cannot become two rows nobody can tell apart.
VLAN numbers had no range check: 99999 and -5 were stored. Worse, a non-numeric
value reached a duplicate probe that compares against an integer column, which
MySQL coerces to 0 - so 'xyz' matched an existing VLAN 0 and the 409 named a row
that had nothing to do with it. 1-4094 per 802.1Q, checked before the probe.
Deleting a VLAN that subnets still reference now answers 409 like every sibling
in-use guard, rather than 400, which told the caller their request was malformed
when the request was fine and the state was the problem.
An appversion sha256 was stored raw, so an over-long value was truncated to the
column width and came back as 64 plausible hex characters - indistinguishable
from a real digest, and wrong. A digest is a claim the delivery path checks
against, so it is 64 hex characters or it is refused. The releasedate goes
through the strict parser for the same reason it did everywhere else.
PUT /api/settings/<key> ignored the declared valuetype. "A6-not-a-number" went
into an integer key verbatim and every consumer then read 0, because
get_typed_value swallows the ValueError - the setting looked set and the
behaviour was the default.
/api/maplevels told a caller who omitted the level name that 'buildingname and
buildingid are required'; buildingname is a field that handler never reads. It
now names the fields it actually wants.
Restoring a guide revision that is already current writes nothing, because
save() dedups on the content hash - and the response still said "Restored
version N as a new draft version", describing a revision that does not exist
while the unchanged revision list disagreed.
GET /api/backups/revisions/<id>/diff?against=<id> accepted any revision id, so
pointing it at another asset's revision rendered that asset's configuration as
the "before" side of this asset's history: a diff that reads as a change on this
machine and describes a different one. Same asset and same kind, or refused.
The default branch was always asset-scoped; only the explicit one was not.
Two frontend fixes: the per-page selector rendered blank on any list defaulting
to a size not in [10,20,50,100] (network devices uses 25), so the operator could
not see the current page size; and an emptied PC alias was dropped from the
payload instead of sent, so it could not be cleared - the form came back showing
the alias that had just been deleted.
Twenty-two tests. The backups guard verified by planting.
Scope: product
Three credential and authz findings, all the same shape: a second path to a
decision, with weaker rules than the path built for it.
PUT /api/apitokens/<id> {"isactive": true} un-revoked a token, and the ORIGINAL
secret worked again - so "revoked" meant "paused until somebody flips it back",
for a credential whose secret is assumed to be out by the act of revoking it.
Revocation is one way now. Deactivating through the same route still works;
bringing one back does not, and the refusal says to mint a new token.
PUT /api/users/<self> accepted a `password` field with no current-password
proof and no length rule, while /api/auth/change-password requires both. When
two routes reach the same state the weaker one is the real rule, so the stronger
one was decoration. Changing your own password through the users route is
refused and points at the right endpoint. An ADMIN may still set one - that is
how a temporary password is issued - with the same 8-character floor, and
mustchangepassword is now set, because a password somebody else chose is
temporary by definition.
tools.secrets was declared guest-grantable, so the role grid offered it and
stored it, and the secure-send route never consulted the guest role at all: it
decides on securesend_allow_guests, which fails closed and carries the rate
limit. Ticking the box changed nothing in either direction, so the screen and
the behaviour disagreed. The declaration is gone; tools.view stays, because it
really does gate the nav entry.
Removing it needed care: update_role refuses the WHOLE request when a submitted
permission is not grantable, and the grid resubmits what the role already holds,
so a site whose guest role held tools.secrets could no longer save that role at
all - the deadlock fixed in 723a4ff, re-created from the other end. Migration
7d41 revokes the grant from the guest role, and nothing is lost because it
granted nothing.
Eight tests. The first two verified by planting: remove either guard and three
fail.
Scope: product
The machine maintenance panel showed every date one calendar day early on this
site, and would show it a day late east of UTC.
lastmaintenancedate and nextmaintenancedate were DATETIME columns holding
midnight, and BaseModel.to_dict serialises a datetime as isoformat + 'Z', so
2026-12-15 00:00:00 went out as "2026-12-15T00:00:00Z" and the browser
converted it into local time. There is no time in a maintenance date: the form
offers a date picker, the CSV importer takes '2026-06-01', and the value is only
ever compared against other dates. Every sibling already models it correctly -
measuringtools.lastcalibrationdate, warranties.startdate and enddate are all
db.Date - so machines were the odd one out.
Migration 7d40 converts both columns; MySQL truncates a midnight time so no
data moves, and SQLite neither enforces nor needs it.
That exposed two more things. The route was assigning the raw request string to
the column, which a DATE column rejects outright - it now goes through the
strict parser from wave 2, so a mistyped maintenance date is refused with a 400
rather than stored or nulled. And BaseModel.to_dict had no branch for a plain
date, so one reached Flask's encoder and came out as an HTTP-date
("Thu, 15 Jan 2026 00:00:00 GMT") - which a browser parses back into a moment
and can still render as the day before. Dates now serialise as "2026-01-15",
with datetime checked first because it is a subclass of date.
Verified against MySQL on a live instance: the column is DATE, the API serves
2026-01-15, a mistyped date is refused with 400, and a rename on a machine that
has maintenance dates saves - the 500 from the audit-log fix in 5d81f50c, now
confirmed end to end rather than only in tests.
Scope: product
The retire dialog says "History and relationships are kept, and it can be
restored." The API always kept that promise - ?active=false lists retired rows,
PUT {"isactive": true} restores one - and no screen in the product could reach
either. A retired asset simply vanished: the lists filtered it out, global
search missed it, and its detail page still rendered Edit and Retire with
nothing anywhere saying it was retired.
Five list pages gained a "Show retired" toggle that passes active=false, and a
retired row now carries a badge. Five detail pages gained a banner saying the
record is retired and what that means, a Restore button, and the suppression of
Edit and Retire while it is retired - offering Retire on an already-retired
record was the part that made the state invisible.
Verified end to end in a browser: retire a machine, the detail page says so and
offers Restore, the list hides it until the toggle is ticked and badges it when
shown, and Restore puts it back (isactive true again on the API).
One thing this does NOT do: global search still does not reach a retired asset.
That is a separate surface with its own filter, and worth deciding on
deliberately rather than sliding in here - a search that returns retired
records by default would change every result page.
Scope: product
Three more findings from the 2026-09-03 run.
Werkzeug answered 405, 415 and a malformed JSON body itself, in HTML. A client
written against the documented {status,data,meta} contract got a page of markup
with no error code, which is indistinguishable from the server having fallen
over. The 405 had a second edge: on a route the caller cannot otherwise see, a
method-not-allowed CONFIRMS the path exists, so a gated kiosk route meant to
read as absent announced itself by refusing the wrong verb. All three now
answer in the envelope, and only under /api/ - everything else is the SPA and
keeps the framework's behaviour, deep links included.
The Toner Report rendered "0 Printers Checked / 0 Low / 0 Critical" and "No
printers match the selected filter" while the API it had just called said
supply monitoring was unreachable. Zeros read as a clean bill of health for a
fleet nobody has looked at. The page now carries the reason the API gave and
suppresses the counts, because a number nobody measured is not a result. Same
defect shape as the supplies card fixed on 2026-08-29; `unavailable` was
already in the payload and the page ignored it.
Backup revisions badged "current" per (asset, kind). NTLARS settings describe a
MACHINE but are read off the controlling PC, so a machine with two reporting
PCs has two independent chains of the same kind - and only the higher revision
id was badged, so the second PC's newest revision showed as history and lost
its last-verified stamp. The machine looked like one source re-reporting rather
than two. sourcehostname is the chain; a revision without one forms its own,
which keeps the old behaviour for everything that has only ever had one source.
Nine tests. The envelope ones verified by planting: remove the three handlers
and three fail.
Scope: product
The driver form and the API both dropped `drivername` and `vendorid`. POST
returned 201 with both null, PUT accepted them and stored nothing, and no
screen offered either field - so there was no interface in the product that
could set them.
That is not a cosmetic gap. `drivername` is the exact string the INF declares,
and Add-PrinterDriver matches on it; `name` is ours to choose and means nothing
to Windows. With it null the shipped converger has nothing to match and
installs nothing, and the drift check that compares an observed driver name
against the expected one could never fire, because the expected value was never
stored. `vendorid` is how a universal driver (HP UPD, Xerox GPD) attaches to a
whole make instead of needing a near-duplicate row per model.
Both are now read on create and update, and the driver form gained the two
fields with hints saying what they are for - the INF name box says plainly that
leaving it blank means the converger cannot install that driver. A whitespace
value clears rather than stores: an empty box means "not known", and a blank
string would look set and match nothing.
Three tests. Verified by planting: drop the two fields from the create path
again and the round-trip test fails.
Scope: product
Three map and aggregation defects. The first is a behaviour change you asked
for; the other two are the 2026-09-03 run's findings.
A dualpath secondary bay was excluded from /api/assets/map unconditionally
while the collapse setting is on, so positioning it was a silent no-op: the
form took the pin, stored mapx/mapy, and the map never drew it. One marker per
pair is right when the pair sits in one place, and that stays the default - but
a dual-bay machine whose second bay is physically elsewhere is exactly what
positioning it separately means. Now: a secondary with its own position draws
its own marker; one without is represented by the primary's, which carries the
combined '2007 / 2008' label. Nothing extra is needed for the fallback, because
the feed already requires mapx/mapy - an unpositioned bay never reaches the
loop.
The combined label is now suppressed when the partner draws its own marker.
Labelling two separate pins '2007 / 2008' would say one machine is in two
places, which is worse than either bay being unlabelled.
Verified against a live instance: unpositioned secondary gives one marker
labelled with both bays; positioned gives two markers each with its own name;
clearing the position puts it back to one. That third case matters - a
one-way door would be its own bug.
machines/dashboard/summary applied the dualpath exclusion to total and bytype
but not to bystatus, so the status bars summed to more than the headline above
them, by exactly the number of secondary bays. A breakdown that does not add up
to its own total is an afternoon of reconciling.
The measuring-tool calibrationstatus filter ran AFTER pagination. It is a
derived value so it cannot be a SQL filter, but applying it to the current
page meant asking for 'overdue' with one overdue tool in sixteen returned an
empty page 1, a total of 16, eight pages, and the row itself on page 3. When
that filter is in play the matching set is now built and filtered before
paging - the same tradeoff the old comment named, spent on correct answers
instead of a cheaper query.
Seven tests. The pagination one verified against the committed version, which
fails it.
Scope: product
Wave 6: two settings pages that told an administrator the site was configured
when it was not. That is worse than a missing feature, because nobody goes
looking for it.
Audit retention had TWO keys. shopdb/core/services/auditretention.py reads
`audit_retentiondays` (category site, default 0). A second `audit_retention_days`
was seeded under category audit with a default of 90, and the Audit & Logging
page wrote to that one - so an administrator set a period, reloaded, saw it
persisted, and nothing ever consulted it. Retention stayed off. The page now
writes the key that is read, the duplicate is out of the seed list, and
migration 7d39 carries a value an operator actually set on the dead key over to
the live one before deleting it: their intent was "keep 30 days", and the fix
should not silently discard it. The page's copy also claimed logs "will be
automatically purged", which nothing does - purging is manual, from the site
settings page or `flask audit purge --apply`.
The Authentication page offered a full SAML configuration - metadata URL, entity
id, ACS URL, auto-create, admin group - and reported "Configured - SAML login
available" once two fields held text. There is no SAML anywhere in the codebase:
no login route, no ACS, no assertion handling. SAML IS planned, so the settings
stay and the page keeps them; what changes is that it now says plainly that
nothing consumes them yet, and the status line never claims availability.
The dangerous control there was "Allow Local Login". Turning it off did nothing
at all, so an administrator could believe they had closed local password login
while it stayed wide open. It is locked on until there is another way in, and
says why. CONFIG.md marks the whole block reserved rather than describing it as
working configuration.
Scope: product
Wave 5, the asset forms. Both defects are the same mistake in different
clothes: a form deciding something it should have been told.
Four forms carried a hardcoded asset-type id - const COMPUTER_ASSETTYPEID = 2,
and the same for machines, printers and network devices - to tell
CustomFieldsInputs which type's fields to render. Those ids come from seed
order. On the instance they were written against, computer was 2; on a freshly
seeded database computer is 1 and 2 is network_device. So the PC form asked for
another type's custom fields, rendered them, and dropped everything typed into
them behind a success toast, because the values were keyed to a type the record
does not have. Verified in a browser against a correctly built dist: the form
now requests assettypeid=1 on an instance where computer is 1.
The name is the stable thing - assettype is a fixed vocabulary and the API
already publishes the mapping - so composables/assetTypeIds.js resolves it once
per page load. A failed load leaves it undefined, which renders no custom-field
panel: a missing panel is honest, a guessed id is what this replaces.
Three of the five forms that emit a map position sent mapx and mapy without
levelid. ADR-017 is explicit that a position without its level cannot be drawn
on the right floor plan, so a picked pin saved a null level and rendered
nowhere - while the form displayed "Position: x, y" back, because it kept the
value it had never sent.
The ADR-017 gate should have caught that and did not: it globbed '*.py' only,
so it policed what the server emits and never what the browser sends. It now
covers .vue and .js too, and accepts both quoting styles. Verified by planting:
drop levelid from PCForm again and the gate names that exact line.
Five composable tests, including a fixture with a DIFFERENT seed order - the
case the constants got wrong.
Still open from this area, and deliberately not in this commit: the PC form
writes an INHERITED location back as the record's own. Fixing it needs the API
to distinguish an owned locationid from a derived one, which is a contract
change rather than a form fix.
Scope: product
Three confirmed findings from the 2026-09-03 run, each independent.
GE-Enforce scope names are directory names on the imaging share, and create_scope
checked only that the name was non-empty. "../escape" wrote a manifest one level
above the share root, "../../escape2" two, and "/tmp/anywhere" wherever it liked
- proven by writing the files and then deleting them. The exposure is the
permission split rather than the traversal itself: geenforce.manage creates the
scope and geenforce.publish writes the file, so a manage-only author could aim a
publisher's write at any path that process can reach. Names are now a single
safe path segment, checked at creation AND again at the write (rows predating the
check exist, and the write is the line that acts), with the resolved target
confirmed to sit under the share root.
Agent timestamps were converted with astimezone(tz=None), which means the
SERVER's local zone, not UTC. An agent reporting 17:26:16Z was stored as
13:26:16 on this EDT box and re-served labelled 'Z', while the same row's
server-written lastreportedat stayed true UTC - one row carrying two time bases
four hours apart. The dashboard's 48-hour stability window compares a real UTC
cutoff against these values, so events left it four hours early. Invisible on a
UTC server, which is how it survived, and nothing in the Dockerfile, the compose
files or the Windows deploy pins TZ. AGENT-CONTRACT.md mandates UTC on the wire,
so a naive value is now taken as UTC rather than guessed at.
The three /api/maplevels reads went back to optional-auth. ADR-017 rule 6, the
API inventory (auth "none"), the module's own docstring and the comment in
PrinterInstallerMap.vue all say they are public, because the printer installer
map runs before anyone logs in and needs a blueprint to draw on. afce2437 swept
165 genuinely-open GET routes behind require_permission and caught these in the
net, so the anonymous installer map has drawn no floor plan since. Restored
rather than amending the ADR: blueprint paths and level names are not secrets,
and the asset positions that go on top already render on public kiosk pages.
The writes stay gated, and a test proves it.
Registered in PUBLIC_ENDPOINTS with the reason. That gate checks a public route
is DECLARED public; the new test checks it actually answers, which is the half
that was broken.
Ten tests. All three verified by planting: restore the non-empty scope check and
eight fail, restore the local-zone conversion and two fail, put the permission
back on the level list and one fails.
Scope: product
Wave 4, second half. PUT /api/machines/<id> returned 500 and rolled the whole
edit back for any machine with a maintenance date recorded - including a rename
that never touched the date. The Machine edit form could not save such a
machine at all, which is most of them once a maintenance schedule exists.
The route builds {'old': getattr(row, field)} straight off the model, so the
changes dict carries whatever the column holds: a date, a datetime, a Decimal.
AuditLog.changes serialised it with a bare json.dumps, which raises on all
three, and nothing caught it. The audit entry and the edit share a transaction,
so the failure took the edit with it.
json.dumps now has a fallback encoder: dates and datetimes become ISO strings,
Decimals become floats, sets and bytes get something readable, and anything
else becomes its repr rather than an exception. An audit entry that reads a
little awkwardly is worth more than a failed write - the alternative, and what
happened, is losing the change AND the record of it.
Fixed at the audit log rather than in the machines route: every audited route
builds its changes dict the same way, so the next one to audit a date field
would have hit the same wall.
Four tests, including the exact shape the machines route produces. Verified by
planting the bare dumps back: two fail.
Scope: product
Wave 4, first half. A personal API token scoped to endpoints.command alone
could queue system_restart, run_script and update_app against a shop-floor PC.
require_permission has always suspended the admin bypass for a scoped token,
and tests pin that. What nothing covered was a route checking a SECOND
permission by hand: the endpoints command route asked
current_user.haspermission(elevated), which reads the token's OWNER. An admin
minting a narrow token for a script therefore handed out remote restart, script
execution and app deployment - the exact bypass require_permission exists to
prevent, reintroduced one hand-rolled check at a time.
caller_holds is now scope-aware, so the answer is about the caller rather than
the person behind it: the owner must hold the permission AND, when the request
carries a scoped token, the scope list must name it. An unscoped token is the
person and is unchanged; a session is unchanged; guests still answer from the
guest role.
Both hand-rolled checks in the tree now use it - the endpoints elevated gate and
the secure-send tools.secrets check, which had the same shape and the same
consequence at a smaller scale.
Two tests, one proving the refusal and one proving an unscoped token still
passes (a fix that quietly broke every service token would be worse than the
bug). Verified by planting the owner-based check back: the first fails.
Reported by the fleet-agent lane of the 2026-09-03 test run and confirmed by an
independent verifier that minted its own token and queued its own commands.
Scope: product
Wave 3. Six findings, one cause: `int(request.args.get('typeid'))` with nothing
catching the ValueError. The caller sent a bad argument and got
500 INTERNAL_ERROR plus a traceback in the log, which says the server is broken
when the truth is that 'abc' is not a number.
shopdb/utils/queryargs.intarg reads it and raises QueryArgError, which an
app-level handler answers as a 400 naming the argument. App-level on purpose: a
route should not have to remember, and forgetting was the entire defect. The
policy differs from get_pagination_params, which falls back to a default for a
bad page number - a nonsense page is harmless and the caller still wants their
list, but silently ignoring a nonsense FILTER returns rows they believe they
excluded.
The gate matters more than the six fixes. tests/test_core/test_bad_query_args.py
sweeps all 131 registered no-parameter GET routes under /api/ with twelve
argument shapes and fails on any 5xx. It found sites the test run had not
reported - statusid and locationid on /api/assets, the second asset-list route,
limit on the employee lookup and the printer supply history - because the
pattern is how a list filter gets written rather than a property of any one
route. Verified non-vacuous by planting the old `int(...)` back into
/api/machines: four tests fail.
Four usb routes are skipped by exact path, with the reason: they reach the
EXTERNAL cmmc_usb database, so their 500 in a test environment is a missing
database rather than an argument fault. That is the complete set - every usb
GET route calling cmmc_usb_connection - not a convenient subset.
One correction worth recording: the first pass rewrote every
`if x := request.args.get(...)` in the touched files, including text arguments,
so a search for 'ZZNOTHINGMATCHES' briefly became a 400. Caught by
test_list_search_identifiers. Only id-shaped arguments are integers.
Scope: product
Wave 2 of the remediation plan. Eight findings from the feature test run were
the same defect twice: a write that reports success while storing something
other than what was sent.
The first half is a date parser that swallowed its own error. Two plugins
carried an identical private _parse_date returning None on ValueError, so an
unparseable date and a deliberately cleared one produced the same write.
PUT /api/warranty/<id> with enddate '30-06-2027' answered 200 "Warranty
updated" and stored NULL, dropping the record out of every expiry surface on
the strength of a typo; the measuring-tool calibration date did the same and
flipped the tool to 'unknown', out of the calibration report and the map
overlay. shopdb/utils/datevalues.py now holds one strict parser on the contract
surface: an explicitly empty value clears the field, anything that is not a
date is refused with a 400 naming the field, and those are different intents
with different answers. The vendor-lookup paths keep a lenient wrapper on
purpose - a provider changing its date shape must not fail the refresh of an
otherwise good record, and no person typed that value.
The second half is why the truncation findings were possible at all. MySQL was
running non-strict: sql_mode on the 5.6 server is NO_ENGINE_SUBSTITUTION with
no STRICT_TRANS_TABLES, so it accepted writes it could not store and adjusted
them. A 257-character username became its first 100 characters (and the next
distinct one then collided with it); an over-long asset number was cut at 50; a
malformed releasedate became '0000-00-00' and permanently broke every read of
that application. All with a 201. The mode is now set on the CONNECTION rather
than left to the server, so a site whose my.cnf disagrees still gets it.
That trades silent corruption for a DataError, which uncaught is a 500 - and a
500 says the server is broken when the truth is that the value does not fit. An
error handler maps it to a 400 naming the column where MySQL names one.
Verified by hand against MySQL 5.6: POST /api/users with a 257-character
username now answers 400 "That value is too long for username." and stores
nothing, where it previously stored a truncated row and answered 201.
Twelve tests. The date guard is verified by planting: restore the swallowing
parser and four of them fail. The DataError tests raise the error directly
because the suite runs on SQLite, which does not enforce VARCHAR length at all -
there is no value that would trigger it there, which is precisely why this went
unnoticed.
Scope: product
Wave 1 of the remediation plan for the 2026-09-03 feature test run. Two kiosks,
one shape of defect, one critical consequence.
Both kiosks read the state, decided in Python, and then wrote. Nothing stops two
requests interleaving between the read and the write, so both see the same
pre-state and both proceed. Reproduced on MySQL 5.6 over HTTP, five concurrent
takes of 5 from a bin holding 5: all five were accepted and quantityonhand
landed on -20, because each request wrote its own arithmetic. The ledger had
recorded every one of them, so the cached quantity and the ledger disagreed
permanently, which is the number the shop floor actually reads. The USB kiosk
had the same shape: concurrent scans of one drive all succeeded and inserted
several open custody rows, so one drive was recorded out to two people and the
documented 409 never fired.
The comparison now lives in the WHERE clause, where the database settles it at
write time. printedparts._move_quantity issues
UPDATE ... WHERE quantityonhand >= :n and treats rowcount 0 as the refusal,
raising InsufficientStock, which take and adjust turn back into the message they
already gave - carrying the quantity the database holds now, not the stale one
the request read. The low-stock alert derives its before-value from what the
write produced for the same reason. Both USB paths claim the device with
ischeckedout (self-hosted) or status (external cmmc_usb) in the WHERE clause;
the external one claims BEFORE writing the log line, so a refused claim leaves
nothing to undo. Check-in is guarded the same way.
After the fix, the same five concurrent takes: one accepted, four refused,
quantityonhand 0 and the ledger sum 0.
Check-in also closes EVERY open row for the device rather than the newest.
One device can only be out once, so a second open row is corruption, and
closing only the newest left the older ones open forever - the device then read
as returned while its custody history said otherwise.
Separately, usbcheckouts now records who handed a drive back
(checkinsso/checkinname, migration usb0004checkinidentity). Returning for a
colleague is a supported flow that the kiosk asks to confirm, but the row it
wrote attributed the check-in to the HOLDER, so the custody history and the
audit report both named somebody who was not there. Nullable and deliberately
not backfilled: copying the holder's badge into old rows would manufacture
evidence for the exact claim this stops making.
Four tests. Both guards verified by planting: strip the condition out of either
UPDATE and its test fails.
Scope: product
Five findings from the review of the WINC import, plus the rendering each fix
needed to be usable.
An undo that hit a conflict wrote status 'undone' anyway, and undo_batch returns
immediately for an undone batch. So the one entry that needed a retry was the
only one that could never get it: the human resolves the conflict, clicks Undo,
and gets a no-op. A conflicted undo is now 'partially-undone', undoneat is
stamped only when the batch is genuinely finished, and the batches table offers
"Retry undo" for that status. The column was String(16) and the new value is
exactly 16 characters, so migration 7d38 widens it to 32 rather than rely on an
exact fit.
_UNDO_REFERENCES listed only assets, so a business unit this batch created was
hard-deleted even when something else still pointed at it. Two of the three
missing referrers are real FKs (dashboarddefaults.businessunitid,
businessunits.parentid): MySQL raises and the whole undo rolls back, so the
batch cannot be undone at all. The third, notifications.businessunitid, has no
FK, so the delete succeeded and left a notification pointing at an id that no
longer exists - silently, on both engines. All four are listed and the guard is
tested through the route.
The reference guard also queried tables that need not exist. A lean site drops
the tables of plugins it does not install (ADR-014), and notifications is
plugin-owned, so on such a site the query raised 1146, rolled the undo back and
left the batch permanently un-undoable. Absent tables are skipped; a table that
does not exist cannot hold a reference.
_PACKET_GROUPS reports on location, but apply_plan only ever creates business
units - it sets locationid only when the location already resolved. A plan
needing a new location asked the admin to acknowledge it, got the tick, wrote
nothing, and reported nothing. build_packet now takes the groups the writer can
actually create; anything else needed is 'uncreatable' and blocks at plan time,
where the admin can act on it, rather than dying at commit on a message about
acknowledging something the screen had no control for. The writer refuses the
same case independently. Both the console packet and the import page name the
blocking groups: setting blocking without printing the reason would leave the
banner pointing at nothing.
The web undo route caught ValueError only, so refusing a batch belonging to
another importer surfaced as a 500. That refusal is new too: the journal is a
general primitive and undo_batch does not filter on kind, so any other
importer's batches were reversible through this endpoint, carrying WINC's
references map - which lists none of that importer's tables, making every
reference check pass and every delete unconditional.
Nine tests. The two guards were verified by planting violations: dropping
dashboarddefaults from the references map, and passing creatable=None, each fail
the test written for them.
Scope: product
Six fixes that were ours regardless of what the sister site does. Four came out
of our own reviews; two you reported from the settings pages.
MAX_CONTENT_LENGTH was set nowhere. Every upload route caps its own payload, but
each of those checks runs after the body has arrived: the first touch of
request.form or request.files makes Werkzeug parse the whole multipart and spool
the file to a temp file, so a route can reject the content and still have paid
for the transfer and the disk. Three separate reviews found that shape in three
different routes, which makes it a property of the app rather than of any
handler. 128 MB, clearing the 64 MB agent binary with room for the envelope.
The naming gate excluded devnotes with --exclude-dir, which matches a directory
of that NAME at any depth - so a force-added plugins/<plugin>/devnotes/ was
invisible to every check in the file, ADR-015 included. Replaced with a path
anchored post-filter. Verified by planting the same site literal in root
devnotes/ AND in plugins/endpoints/devnotes/: the gate now reports exactly one,
the nested one. Before, it reported none.
BaseModel.to_dict walked __table__.columns and published every one, so a model
without an override published whatever it carried - including columns added
later by someone not thinking about this method. User has no override, so
User.to_dict() returned its bcrypt hash. Nothing calls it in a response path
today, which is luck rather than design. A blanket allowlist would empty the
output of all 34 models relying on the default, so this pins a floor instead:
named secret-bearing columns are never emitted, and a model may extend the set.
Eight tests, including a sweep over every mapped model.
ActionPicker's Esc handlers had .prevent but not .stop, so the event bubbled to
the wrapper and closed the whole picker instead of returning to the monitor list.
The branding page reported "error loading image" on the site and badge logos.
Two bugs stacked. The load failure was real: the preview used the stored value
as a raw src, and that value is root-relative, so under /shopdb or /ops it
resolved against the SERVER root and 404d - siteSettings.js wraps every logo in
withBase() and this page was the one place that did not. With that fixed two of
three were still invisible, because the shipped GE mark is fill="white" for the
dark sidebar and was rendering white-on-white. The preview tile now sits on a
mid-tone checker so artwork of any colour reads against it.
The floor-map levels table ran off the right with its cells crushed - nine
columns of inputs, no scroll wrapper, and .data-table is styled nowhere. My
first attempt reached for the global .table-container and made it WORSE: that
class also sets white-space: nowrap, and a full sentence of hint text lives in
one of the cells, so nowrap forced the table wider still. It now scrolls in a
local wrapper, the hint wraps, and the name input has a width instead of
collapsing to one character.
Suite 2377, frontend 324, gate green.
Scope: product
One commit from the sister site, purely additive: a read-only `batches` command
so an operator working from the shell can find a batchid to undo, and a README
covering the four WINC commands. Two new files under scripts/site_imports/winc/,
nothing existing touched.
Verified rather than assumed: it changes none of the files fixed in 13be0ef9
(the workflow YAML, the approval check, the agentpackage description, the upload
audit), the gate passes, all five workflows still parse, and the project map is
current.
Suite 2369.
Scope: product
Eleven commits from the sister site: a reversible Maximo/WINC machine-adoption
import, a versioned agent binary store with upload and rollback, remote-BuildKit
CI, tray version reporting, and UI fixes. Applied as a patch again - the
published mirror rewrites hashes, so the branches have unrelated histories and
git refuses to merge them.
Reviewed by three agents before merging. Four things fixed here.
THE PUBLISH WORKFLOW WAS DEAD. The new remote-BuildKit step in
publish-images.yml sits at column 0 instead of indented under steps:, so the
whole file is invalid YAML and GitHub would refuse it. Their entire GHCR
distribution channel - which Start-ShopDBDev.ps1 and the compose files consume -
was broken at the source and clearly never run. Indented; all five workflows now
parse.
THE IMPORT'S APPROVAL CHECK COULD NOT FAIL. commit() built the plan and the
approval packet from the same upload and then asked the writer whether they
matched, so packet_matches_plan compared a hash against itself. The documented
contract - "a human approves a specific, frozen plan, and the writer can prove
nothing drifted" - did not exist in the web path: an admin could preview file A
and commit file B, gated only by the needs-create group NAMES they ticked. The
preview's planhash is now required at commit and checked against a freshly
rebuilt plan. Their tests committed without previewing, which is what let this
through; they go through the real flow now, and three new tests pin it - all
confirmed to fail with the check stubbed out.
endpoints.agentpackage QUIETLY GREW. It used to mean "wrap the staged exe in a
ZIP"; with the upload routes it also means "choose the executable every future
package installs and runs as SYSTEM". Before this, staging needed filesystem
access to the server. The permission description still said only "generate and
download a package", so an admin granting it could not see what they were
granting. Reworded.
NOTHING RECORDED WHAT WAS STAGED, OR BY WHOM. The manifest held version, size
and timestamp; re-uploading a version replaces the bytes in place under the same
label. A sha256 of the stored file is now recorded, and the upload is
audit-logged with it - after an incident there was previously no answer to who
put that binary on the fleet.
Also documented the nine new routes in docs/api-inventory.json (that gate cannot
run on the mirror, so it falls to us at promotion time again), regenerated the
spec and the project map, and kept our own docs where they conflicted: docs/ is
scrubbed from the mirror, so their DEV-BUILD-TEST.md still had the dev-be
channel mapping we corrected, and their PLUGIN-HOOKS.md was a new 45-line file
rather than our 737-line canonical one. Their journal section is appended to
ours.
Contract version comes across as 0.30.0 for a genuinely additive surface.
Suite 2369, frontend 324, gate green.
Scope: product
The USB kiosk had a feature flag and nothing else. A flag is a decision, not a
credential: with the kiosk on, anyone who could reach the app could move a drive
between checked-out and checked-in and sign it with a badge they guessed. Same
reasoning that closed the parts kiosk, smaller blast radius - which is why
Darkwing did not raise it and why it was worth asking about anyway.
Both kiosks now share one mechanism. The device token moved out of the
printedparts api wrapper into a request interceptor that attaches X-Kiosk-Token
by URL prefix, with one localStorage key per kiosk - they are separate screens
in separate rooms, and a shared token would mean provisioning one grants the
other. claimKioskToken() is shared too, so both views take the token from a
?token= URL once and strip it from the address bar identically.
Then the part that matters more than either fix. I told you to set the new
settings in the UI, and that was impossible: every plugin settings page carries
a hardcoded key list, and nothing had added them. printedparts_kiosk_enabled,
printedparts_kiosk_token and usb_kiosk_token were declared, seeded and read
correctly, and rendered by no page - changeable only by writing SQL.
tests/test_plugin_settings_reachable.py now fails when any declared setting
appears nowhere in its plugin's frontend. It found seven MORE that were already
in that state, all pre-existing:
collector_corporate_ranges computers_configpolicies_enabled
computers_machinelink_alerts computers_quietreporthours
geenforce_reportstaleminutes printers_dashboardpercent
warranty_expiringdays
All seven now have fields. I first parked them in a KNOWN_GAPS ratchet, then
fixed them and deleted the ratchet - a backlog list in a new check is a way of
agreeing the check does not apply yet. There is no backlog; the check is
absolute.
Two of my own mistakes on the way, both caught by tests rather than by reading:
the blanket rename of the USB guard also rewrote the call inside _kiosk_open, so
it called itself until the stack ran out; and the first version of the reachable
check skipped all 17 plugins because it looked for get_settings() when the
method is get_settings_defaults() - it passed by testing nothing, which is the
exact shape it exists to catch.
Suite 2336, frontend 316, build clean.
Scope: product
printedparts_kiosk_token declared valuetype 'secret'. The column documents
string, boolean, integer and json, and that was the only 'secret' in the
codebase - it worked only because Setting.get falls through to the raw string
when it recognises nothing.
Masking never depended on it: settings.py::_is_secret keys on the NAME
containing 'token', so the value is still hidden in API reads and in the audit
log. Changed to string, which is what it is.
Scope: product
Both from the second Darkwing scan. Every claim in it was verified against the
code before acting.
The kiosk was the product's only unauthenticated write, justified by a decision
record saying it was "physically rate-limited: it serves a touch screen on the
shop floor". Nothing in code enforced the physical half, and an HTTP route is
reachable from wherever the app is. What that left open:
- _kiosk_find_item falls back to matching bare digits against the numeric
tail of every active item, so 1, 2, 3 walk the catalogue without knowing a
barcode;
- a badge is a short numeric SSO, and the route answered 422 for an unknown
one and 200 for a real one - an oracle for sorting guesses into employees;
- the write then decremented stock and stamped the ledger, which IS the audit
trail, with whatever SSO the caller supplied;
- quantity was bounded only by what was on hand, so one request emptied a bin.
A badge is attribution, never authorization. Both kiosk routes now need the
site's kiosk flag AND a per-device token, and 404 rather than 403 without them
so a caller does not learn the door is there. The READ is gated too: gating only
the write would have left the enumeration oracle running against a kiosk a site
had switched off.
The token is per-DEVICE, not per-page, because a kiosk is a browser on a touch
screen and a token served inside the page is readable by anyone who can load it.
An admin opens the screen once as /parts-kiosk?token=..., the page moves it into
that browser's localStorage and strips it from the address bar.
Also: quantity is capped per take, JSON true no longer passes as 1
(isinstance(True, int) is True), and every badge failure on the anonymous route
returns one message. The gated restock and adjust handlers keep the specific
message - there the person is known and the detail is help, not an oracle. I
changed the wrong one of those three handlers first; caught by the test.
/api/dashboard/health formatted str(e) straight into the body of a route
PUBLIC_ENDPOINTS declares open because "monitoring has no account". A pymysql
OperationalError names the database account, the app host's address, the driver
and the failing statement. It is logged server-side now and the body carries a
coarse status. The version goes with it - it told an anonymous caller which
release to look up. Status stays 200 either way: a monitor keyed on the body's
status field is the contract here, and turning a degraded database into a
transport failure is a separate decision from not leaking the exception.
Nine tests. The three gate tests were confirmed to fail with the gate stubbed
out, rather than assumed to cover it. Suite 2325, frontend 316, build clean.
Scope: product
The previous commit said the link-resolving change had never met a real
filesystem. Run in the win11 VM against real fixtures, it turned out to reject
EVERY legitimate launch path.
ResolveDirectoryChain walked up looking for a junction and eventually asked
whether "C:\" was a link. That throws DirectoryNotFoundException, and the caller
treats a resolve failure as a rejection, so the first legitimate case came back
"cannot resolve the real path: Could not find a part of the path 'C:\'". The
agent would have failed closed on everything - an outage wearing a security
fix's clothes. It compiled with zero warnings and read correctly.
Stop at the volume root, and never let "cannot tell whether this is a link" turn
into a rejection: TryResolveLink returns null on any failure, and the assembled
path is re-checked against the roots regardless, so an unresolvable component
still cannot widen what is accepted.
One expectation in the probe was mine being wrong rather than the code. 8.3
short names used to be rejected because the lexical prefix check could not match
C:\PROGRA~1 against C:\Program Files - fail closed, but by accident of not
understanding the path rather than by intent. Resolution now expands it to the
real directory, which is inside the approved root, so accepting it is correct.
Verified on Windows, all ten cases:
accept a real exe under the approved root
accept 8.3 short name resolving into the approved root
reject exe reached through a junction pointing out of the root
reject alternate data stream on a file in the root
reject traversal back out of the root
reject UNC path
reject sibling directory sharing the root's prefix
reject wrong extension
reject file that does not exist
reject empty path
The probe is committed as agent/ShopDbEndpointAgent.PolicyProbe so this is
repeatable. It is NOT in CI and its README says why: junctions, alternate data
streams and 8.3 names do not exist on Linux, and a suite that skipped every case
it exists for would report green and mean nothing. This repo has been bitten by
that shape more than once already.
Scope: product
check to the file it actually runs
Two clusters the five-agent review found. Both are about a product that another
plant has to be able to install.
ADR-015: the shipped monitor bundle was the originating plant's fleet
convention frozen into product seed.
The business policy was seeded as "RPCA Shopfloor App Monitors". RPCA is that
plant's facility designator, the row is source='shipped', and the API refused
to rename it - so every adopting site would carry another plant's label at the
top of its policy list forever. 0007 now seeds the neutral name, and 0010
renames it on installs that already stamped 0007, guarded so it cannot collide
with a policy a site made itself. Downgrade is one-way on purpose: putting
another site's designator back is not a state worth reaching.
Which PC type the bundle attaches to was the literal 'Shopfloor PC'. A plant
whose type is "Shop Floor Workstation" silently got no shipped monitors at
all, and one using that name for something else got all of them. It is a
setting now, defaulting to the seeded name so nothing changes here, and a
blank value attaches the bundle to nothing - the right neutral answer for a
site that does not want it.
Shipped rows refused every edit, including isactive and launchpath. A plant
whose eDNC is on D:, or that runs no UDC, could not correct the path or switch
the monitor off, and then carried permanent false drift for it. ADR-015 allows
seed data BECAUSE a site can edit it afterwards. Paths and the active flag are
now editable; kind and displayname stay fixed, because that pair is the
identity 0007's bundle matches on.
Also corrected two false claims in 0007's docstring: the policy tables belong
to the computers plugin, not the core baseline, and a guarded no-op does NOT
reconcile later - Alembic stamps the revision either way, so the bundle would
simply never seed.
LaunchPathPolicy was purely lexical. GetFullPath is string normalization that
never touches the disk, so the root check constrained the NAME while File.Exists
and CreateProcess followed reparse points to somewhere else. A junction under a
writable directory in an approved root - creating one needs no privilege, only
write access to the parent, and vendor trees under Program Files are not
reliably locked down - turned WebUI monitor-edit rights into code execution as
the console operator. It now resolves the real target, leaf and directory chain
both, and re-checks THAT against the roots, rejecting if resolution fails.
Alternate data streams passed the whole policy on net8.0: an interior colon no
longer throws in GetFullPath, so "readme.txt:payload.exe" reported an .exe
extension, passed the root check, and existed. Rejected explicitly now.
Six tests added for the shipped-row and pctype behaviour. The Core project
compiles clean. The launch-path change has NOT been exercised against real
junctions, ADS or 8.3 names - that needs the win11 VM, and per the reviewer it
wants a planted-violation suite there rather than my say-so.
Still open from the review: monitor revocation does not take effect until agent
restart, ActionPicker is keyboard-inoperable, the lightbox has no focus trap,
one attached monitor grants generic any-target process control, and screenshots
are retained unencrypted in the user profile.
Scope: product
Findings the reviewers verified, fixed here. The larger cluster they found is
NOT fixed and is listed at the end.
Frontend. reportImageUrl is called from the template, so it ran on every render
with no in-flight guard: a report whose fetch was pending was re-requested, and
because each resolution reassigns the map the whole grid re-rendered and
re-fired the rest, costing up to N-squared requests for N thumbnails. Two blobs
resolving for one key orphaned the first object URL where revoke could not
reach it, and a failure was never recorded so a broken image retried forever.
Guarded, deduplicated, failures remembered.
Three CSS variables did not exist, so their hardcoded fallbacks always won:
--bg-alt (the "No screenshot" tile, near-black in light mode), and
--warning-text/-bg/-border (dark khaki on a dark modal). Replaced with real
tokens. The DEV badge's literal #c62828 is var(--danger).
Installer and launcher. The generated site installer ran docker compose with no
-p, so the project name defaulted to the package folder - which carries the
version. Installing the next version created a new project with new empty
volumes and the site came up looking wiped, old data stranded under the old
project. Pinned to shopdb-<channel>.
Start-ShopDBDev deleted any existing -TargetDir that lacked a .git, recursively
and without asking. For a script handed round as "save it anywhere and run it",
a mistyped -TargetDir pointed at Documents would have destroyed it. It now only
reclaims a directory that is empty or carries this launcher's own marker, and
refuses otherwise.
Dev secrets came from Get-Random, which is System.Random: a 32-bit seed, and all
four values off one stream, so recovering one recovers the rest. They sign JWTs
for a stack a pilot may point at real imported data. CSPRNG now, and the API
port binds to loopback like the DB rather than every interface.
Two of my own from earlier in this session. The operator-report size check sat
below request.form, and the first access to that parses and spools the whole
multipart - so it ran after the thing it prevents. Moved above. And the channel
remap missed build-channel-installer.ps1 entirely, where it set BOTH channels to
main: a "stable" package built from the trunk, the exact mislabel the remap was
for. stable now resolves the newest v* tag.
Also removed a zero-byte docs/ShopDB-Planner-Tasks-v2.csv that rode in.
NOT FIXED - needs a decision, and it is the biggest thing the review found.
Migrations 0006/0007 freeze the originating site's fleet convention into
product seed: a shipped policy literally named "RPCA Shopfloor App Monitors",
a hardcoded 'Shopfloor PC' type deciding who gets the shipped watch-list, and
shipped rows the API refuses to edit or deactivate - including their launch
paths. A site whose eDNC is on another drive cannot correct it and gets
permanent false drift. Those are stamped migrations, so this gets harder to
undo at every install. See the review notes before anyone upgrades a second
site through 0006-0007.
Suite 2314 passing, frontend 316, all three PowerShell files parse.
Scope: product
Six commits from wilm-import: the squashed import, route documentation, the
channel remap onto our tag-and-trunk model, the project map, parallel test
support, and the four smaller review findings.
Merged rather than cherry-picked: the published mirror rewrites hashes, so the
commits do not apply one at a time against our history even though the squashed
diff applies clean.
Scope: product
# Conflicts:
# docs/PROJECT-MAP.md
None blocked the merge; all four were mechanisms that did not mechanise.
_RUNNING_STATES existed twice, in api/routes.py and services/reconciler.py, with
a comment asking a human to keep them in lockstep. The panel and the reconciler
read the same agent snapshot, so a drift between the copies would paint a
service green and flag it as drift on the same page. One definition now, owned
by the reconciler and imported by routes - api -> services is the direction the
plugin layout already runs in, and the reconciler's import of routes is deferred,
so there is no cycle.
The operator-report upload checked its 10 MB cap AFTER upload.read() had already
pulled the whole body into memory, so the check could not prevent the thing it
was there to prevent. content_length is now an early out and the read itself is
bounded to cap+1, which is enough to know the cap was exceeded without taking a
body that lied about its length. The multipart envelope carries the other form
fields too, so the early out allows 64 KB of slack rather than rejecting an
image that is just under.
The same handler forced the extension to .png and the mimetype to image/png
whatever the bytes were, then served the file back under that type. The upload
headers (CSP sandbox + nosniff) make the document case inert, so this was never
the hole it looks like, but nothing is gained by storing a file we already know
is not what it claims. The PNG signature is now checked.
The lightbox bound Escape and the arrow keys to a div and never focused it, so
every one of those bindings was dead until the user happened to click the
backdrop. It takes focus on open and hands it back to the thumbnail on close.
On the two upload tests, honestly: the PNG one was confirmed to fail with the
check removed. The oversize one passes either way, because the old code did
reject an oversized image - it just did so after reading all of it. That test
pins the response, not the fix. Whether the read is bounded is not observable
through the response, so nothing here proves it; the change stands on reading
the handler.
Suite 2311 passing, frontend 316 passing, build clean.
Scope: product
The suite had grown to ~2300 tests and 5m30s. Profiling says there is no hot
spot to fix: the 25 slowest tests sum to about 60s of 331s, and the rest is
per-test fixture cost. The db fixture drops and recreates all 94 tables for
every test, measured at 0.058s a cycle, which is roughly 40% of the wall clock
on its own.
That shape parallelises almost linearly, so pytest-xdist buys more than any
rewrite would:
pytest -q 2309 passed in 331s
pytest -q -n auto 2309 passed in 103s (8 cores)
Deliberately NOT in pytest.ini addopts. Worker startup is dead weight on the
targeted single-file runs that make up most of a working day; the whole-suite
run before a push is where it pays. Opt in with -n auto.
A run distributed across 8 workers also has to be order-independent to pass at
all, so this is incidental evidence that the suite has no inter-test coupling.
Recompiling requirements-dev.txt to add xdist also moved idna, packaging and
pyjwt. That is the -c requirements.txt constraint doing its job rather than a
drift: all three ARE shipped, and the dev file had fallen BEHIND production
(idna 3.13 in dev against 3.19 in prod). CI was testing against older versions
than sites install, which is the exact failure the header comment on that file
was written about. Now matched.
Scope: product
The channel work assumed the branch model it was written against: upstream,
main is a release branch and dev-be is the trunk. Neither holds here. main IS
our trunk - 212 commits past v0.11.3 - and a release is a tag (ADR-007). dev-be
does not exist at all, so the edge half would never have fired and the launchers
would have cloned a missing branch.
Taken literally it would have published our unreleased trunk to every site as
:stable, which is the opposite of what the word promises to someone who has no
way to check.
Remapped throughout:
tag v* -> :stable the thing that was actually released
main -> :edge the trunk, DEV badge lit
publish-images.yml keys the channel on github.ref_type = tag rather than on a
branch name, so the mapping cannot silently drift back. Start-ShopDBDev.ps1,
dev-bootstrap.ps1 and build-channel-installer.ps1 resolve stable to the newest
v* tag via git ls-remote --sort=-v:refname, warning and falling back to main if
a repo has no tags yet. DEV-BUILD-TEST.md follows.
The push: trigger is commented out. Automatic publishing waits on someone
confirming the GHCR package exists under this owner and deciding who may pull
it; workflow_dispatch works meanwhile. Turning it on is one uncomment.
All three scripts parse clean under pwsh, and the tag-selection expression was
run against this repo rather than eyeballed. Per the standing rule, the runtime
paths still want a pass in the win11 VM before anyone leans on them.
Scope: product
Darkwing's reconnaissance flagged POST /api/printedparts/kiosk/take as an
unauthenticated write reaching a ledger commit and an outbound mail/webhook
trigger. Analysis dropped it, correctly: it is a declared exemption with a
decision record. Nothing to fix in the code.
The comment beside it was wrong, though. It described the USB kiosk writes as
open at the "same bar as the parts kiosk" while listing a gate the parts kiosk
does not have: usb_kiosk_enabled, off by default, every route a 404 until a site
sets it. grep -rn kiosk_enabled plugins/printedparts/ returns nothing.
So a site that adopts the product and has no parts kiosk still serves a live
unauthenticated inventory-decrement endpoint with no switch to turn it off, and
the registry that is supposed to be the record of these decisions said the
opposite. Whether printedparts should gain the same flag is a behaviour change
for a kiosk that is in daily use here, so it is left as a decision rather than
made silently; this only stops the comment misleading the next reader.
Scope: product
Found by the Darkwing scan of main (run 33654922078). Its own review dismissed
the finding as not remotely exploitable and it is right about that: nothing can
make a victim's browser attach a custom header, and deploy/windows/web.config
does no output caching. It still recommended the hardening, and the reasoning
for the dismissal is a property of OUR IIS config, not of the product - every
site runs its own instance behind its own proxy under ADR-004.
_batch_base_url() interpolated a caller-supplied host into the single-quoted
PowerShell -Uri literal of the .bat this route hands out. site_base_url ships
blank, so that fallback is the default path on a fresh install, not an edge
case, and GET /api/printers/install-batch is public.
Three rules now:
- the host is validated as a hostname before it is interpolated, so the quote
that would end the literal never reaches it,
- X-Forwarded-Host is only believed when TRUSTED_PROXY_COUNT declares a proxy
-- the rule shopdb/utils/clientip.py already applies to X-Forwarded-For.
That was centralised for this exact reason and did not cover the Host
variant,
- a request supplying neither usable header gets a relative base, which still
downloads correctly from the browser that fetched the script.
Three tests pin it, and all three were confirmed to fail with the old body
restored rather than assumed to cover it.
Scope: product
Two of the three things holding the wilm import out of main.
The monitor CRUD, the two resolver reads and the three operator-report routes
had no entry in docs/api-inventory.json, so test_api_inventory_parity failed.
The gate cannot run on the published mirror - the test ships but its data file
is scrubbed, and it skips on the file's absence - which is why the branch that
added the routes never saw it. Cost lands here at promotion time instead.
/api/endpoints is registered both with and without a trailing slash and both
forms are served, so both are described rather than guessing which one counts.
Spec regenerated: 353 paths, 504 operations.
plugins/endpoints/plugin.py had a contiguous run of continuation lines shifted
by one space. Python accepted it and git diff -w showed nothing, but it put
deletions in a diff that is otherwise purely additive. Restored; the file now
differs from the published main by additions only.
Scope: product
GE Aerospace now expects a Darkwing check on every repo. Two files rather than
one, because they answer different questions and only one of them is ours.
scan-darkwing-portable.yml is copied byte-for-byte from the ge-aero/darkwing
README so it keeps matching the sanctioned integration. Manual dispatch only.
scan-darkwing-pr.yml runs on pull requests and is ours to change. It exists
because the README workflow cannot simply be given a pull_request trigger: it
resolves its target as `inputs.target_ref || 'main'`, and a pull_request event
carries no inputs, so the expression falls through to 'main' and the job scans
the default branch instead of the code under review. That is a check that is
green because it never looked. This one resolves the PR head SHA explicitly.
Neither file gates a merge. Darkwing exits 0 whatever it finds, and its results
vary between runs on identical code, so a passing job means the scan ran and
nothing more. The SARIF-parsing step that would turn a finding into a failure is
written but commented out, to be switched on once the false-positive rate on
this codebase is known.
Scope: product
Everything on origin/wilm that is not on the published main, applied as one
change because the published history has rewritten hashes: per-commit
cherry-pick conflicts against our main even though the squashed diff applies
clean.
Contents: endpoint monitor catalog + policy attachment + warn-only drift
reconciler, operator issue reports from the tray, agent launch-path allowlist,
and the GHCR channel-image distribution path for dev workstations.
NOT READY FOR main. Open items:
- 10 new routes are absent from docs/api-inventory.json, so
test_api_inventory_parity fails. That gate SKIPS on the published mirror
(the test ships, its data file is scrubbed), which is why wilm never saw it.
- dev-bootstrap.ps1 and Start-ShopDBDev.ps1 hardcode the ge-aero GitHub org.
- plugins/endpoints/plugin.py carries a whitespace-only re-indent.
Scope: product
Eight fields across three asset types were shown on the detail page and
accepted by the API, but had no input on the form. A wrong value therefore
needed database access to correct, which is not a thing anybody should need
for a device category or a maintenance date.
The shape is the same in each case: a column was added, the detail card was
written, the API allowlist was updated, and the form was missed. Nothing was
broken - the values arrive by import or collector - so nothing complained.
PCs gain azureobjectid and azuredevicecategory. azuredeviceid was already
there, which is what made the omission easy to miss: the page showed two ids
and the form offered one. The two are not interchangeable and the pairing is
counterintuitive - the DEVICE id opens Intune, the OBJECT id opens Entra - so
each input says where it goes.
Machines gain the three maintenance fields. These matter beyond the immediate
fix: they are the columns a measuring instrument's calibration dates map onto
when metrology moves to machines, and until now there was no way to enter a
calibration date on a machine at all.
Printers gain iscolor, isduplex and isnetwork as a capability group.
isnetwork defaults to true in the form to match the API's create default; a
new printer claiming otherwise would be wrong for nearly all of them.
Checked before making any of these editable that nothing writes them
automatically - a hand edit silently reverted by the next collector poll would
be worse than no input at all. The printers collector reports queues and
supplies, not capabilities, and nothing writes machine maintenance dates,
which is exactly why they are all empty. The two genuinely collector-owned
computer fields, lastreporteddate and lastboottime, are deliberately still not
editable.
Scope: product
A CMM reported 100.0.0.1 as its address and never reported 10.48.255.210 at
all. Both halves of that were the same mistake.
THE BAY WAS CHOOSING. The collector schema took a single `ipaddress`, so the
script picked one NIC and dropped the others - preferring an address inside a
configured range, else whichever interface carried a default route. A bay
cannot make that judgement: it does not know the site's addressing. On this
CMM the Marvell AQtion 10Gbit link to its controller is a real hardware NIC,
up, and carries a default route of its own, so it satisfied every test the
script had. It won, and the corporate address was discarded before the request
was even built - which is why it is not in ShopDB even as a second row.
So the bay now reports EVERY connected hardware NIC with a routable IPv4, and
takes no view. The server picks the primary from collector_corporate_ranges, a
new per-site setting, and writes a Communication row for each of the others.
Naming the ranges is what turns the answer from a guess into a decision; with
none set it still falls back to the default-route flag and then to the first
reported, both guesses, both better than before only because nothing is thrown
away now.
Rows are keyed on the MAC where the bay sends one, because an address moves
between adapters and a MAC does not. A NIC that is down at check-in is left
alone rather than deleted - being down is not evidence the record is wrong,
and hand-entered gateway, DNS and serial settings live on those same rows.
Exactly one row ends up flagged primary, and only a row this report touched
can claim it, so a check-in cannot silently demote an address set by hand.
-AllowedRanges and the CollectorRanges registry value are now accepted and
ignored, logging a line that says where the setting moved. The parameter stays
because a scheduled task still passing it would otherwise fail on an unknown
argument.
A bay on an older collector sends one address and no interfaces; that keeps
working and is pinned by a test.
Scope: product
Twenty computer types existed and six had a machine on them. Three of the
empty ones were near-duplicates of live types - Shopfloor beside Shopfloor PC,
CMM beside CMM PC, Engineer Workstation beside Engineer - which is the kind of
list that grows because deleting from it by hand is nerve-wracking.
retype-by-role types a PC by WHAT IT CONTROLS. A metrology box has an active
relationship to a measuring_tool asset, so the fact is already recorded and
does not need inferring from a hostname or a tool's name. The imaging scope is
the authoritative source and stays so, but only 17 of roughly 292 PCs have
ever reported one, so it cannot answer for the fleet yet. Relationships can:
49 PCs drive a measuring tool, and 36 of them are currently typed as ordinary
shopfloor machines.
prune-types deletes a type only when nothing is on it AND the live pctype map
does not name it. The second guard is the one worth having: the map is what
the collector assigns from at check-in, so deleting a mapped type would leave
every future check-in logging 'mapped ComputerType not found' instead of
typing the machine.
Both report by default and write only with --apply, and prune prints why it is
keeping each type it keeps rather than only what it removes.
Also fixes `computers stats`, which counted the computer type named
'Shopfloor' - the EMPTY duplicate - rather than 'Shopfloor PC'. It has been
reporting 0 shopfloor PCs for as long as both rows existed, which is the same
duplication these commands clear up.
Scope: product
The Entra Device Info card showed two GUIDs and nothing to do with them.
Copying one into the portal by hand is the kind of errand a detail page exists
to remove.
Each link sits BESIDE the id it is built from rather than in a row at the top
of the card, because the pairing is counterintuitive: the DEVICE id opens
Intune and the OBJECT id opens Entra. A separate action row would have to
re-label which is which, and would still leave the reader matching them up.
BOTH URLS ARE SETTINGS WITH AN EMPTY DEFAULT, and that is the whole design.
The host is a tenant fact, not a product one - this site is on the US
Government cloud, portal.azure.us, where a commercial-cloud default would send
people to a page that does not exist. Microsoft also moves these blade paths,
and a site that hits that can fix it by editing a setting rather than waiting
for a release. Empty means no link and the id still renders as text, so a site
that configures nothing is unchanged. The example values live in each
setting's description, per cloud.
Same shape as the ServiceNow URL templates already in integrations, down to
the {token} placeholder, so there is one idiom for "deep link into a system we
do not own" rather than two.
Scope: product
An Azure device import died part-way with a foreign-key error, and the crash
was the lucky outcome.
The Azure adapter routes a device that already exists to its real asset by
writing that asset's assetnumber into the assetid column of the CSV it
generates, expecting the engine to look it up. The resolver instead takes any
all-digit string at face value as a literal id. This site numbers operations
0NNN, so numeric assetnumbers are ordinary here: `0704` was read as id 704,
which does not exist, hence the error. `0600` would have been read as id 600 -
a printer - and bound a PC to it silently, reporting success.
Two changes, both needed.
The resolver now tries the label first for an all-digit string and only falls
back to reading it as an id when nothing is named that. The face-value
behaviour exists for a real reason - exports round-trip and carry real ids, as
its original comment says - so it stays as the fallback rather than being
removed. Somebody writing `0600` in a column keyed on assetnumber means the
asset called 0600, and binding the row to an unrelated record is far worse than
failing.
_existing_hostname_map now hands over the assetid rather than the assetnumber.
Giving the resolver the id it is going to treat the value as makes the
passthrough correct by construction instead of correct by luck.
Verified against a restored copy of the production database: 234 devices, 221
matching an existing computer and updating in place, 13 new, no problems. The
same run previously failed on the foreign key.
Neither half was wrong alone. The passthrough dates from the original CSV
import and the hostname map from the wilm merge months later; they only meet
at a site whose natural keys are numeric.
Scope: product
"Metrics" promised numbers. The tab holds tracked services, tracked processes
and network interfaces - things being WATCHED and their current state, only
one of which is a measurement. Monitors is what the panels actually are, and
it matches the word the agent's own config already uses for them.
Moved to second, ahead of Software. What is being watched on this box is the
question asked right after "what is it", and well before its application
inventory - which is reference data somebody goes looking for deliberately.
Tab order is now Details, Monitors, Software, Activity.
Contract 0.29.0 keeps its number. It is unreleased, so the group vocabulary it
introduces is still being settled and nothing can yet depend on the old name.
Both the server and the client fall back to 'details' for a group they do not
recognise, so a site that deploys these halves out of step gets a misplaced
card rather than a missing one.
Scope: product
SOFTWARE OPENED COLLAPSED. The installed-software panel was collapsible and
started that way, because it used to be low-priority reference data near the
bottom of one long page. It is now the Software tab's whole reason to exist,
and somebody who clicked that tab has already said what they came for. Made
it neither collapsible nor collapsed.
TRACKED SERVICES VANISHED FROM METRICS. The group is decided by the SERVER, so
a new frontend against a server still serving the old vocabulary puts a panel
in a group the build does not render, and it is fetched, held, and shown
nowhere - with nothing logged. Moving services out of 'actions' is exactly that
shape. The client now falls back to 'details' for any group it does not render,
the same way the server does, so a version-skewed pair degrades to a misplaced
card instead of a missing one.
A TABLE PANEL WITH NOTHING IN IT HID ITSELF. panelVisible honoured `empty` for
list panels and ignored it for tables, for no reason anybody wrote down. A
declared empty message is an answer - "No installed software reported." - where
an absent card is a question. It matters more now that a panel can be the only
thing in a tab, because the tab disappears with it.
THE TAB STRIP HAD A VERTICAL SCROLLBAR. Per the overflow spec, when one axis is
not `visible` the other computes from `visible` to `auto`, so `overflow-x: auto`
alone made the strip scrollable vertically too and the -1px margin the tabs use
to sit on the border was enough to trip it.
Also moved .detail-full into the global stylesheet next to .content-grid, per
frontend/CLAUDE.md: it is detail-page layout, and the other four detail pages
need it when they adopt tabs.
Scope: product
Four changes from looking at the built page.
ACTIONS HELD ONE CARD. Tracked services was the only thing in it, and it was
the wrong split anyway: a service row reads as state first and a Start/Stop
second, so separating the buttons from the state they act on made a tab that
answered no question on its own. Services move to Metrics, beside the rest of
what the endpoint reports, and the group is gone from the contract.
WIDE PANELS SPAN, THEY DO NOT SCROLL. Network interfaces carries adapter,
address, MAC, speed and state; at half page width in the two-column flow it
either squeezed every cell to three lines or offered a horizontal scrollbar.
A scrollbar for data that just needs room is a layout failure. Table panels
now take column-span, which is the multi-column layout's own answer, so short
fact cards stay two-up around them. The wrapper keeps overflow-x only as a
floor for a genuinely narrow viewport.
SOFTWARE AND ACTIVITY RUN FULL WIDTH. Both are single long lists, and a list
split into two balanced columns reads as two unrelated lists with no way to
tell where the first ended. Details and Metrics keep the two-column flow
because they are short cards, which is what that flow is good at.
Contract 0.29.0 keeps its number: it is unreleased, and the group vocabulary
it introduces has simply lost one value before anybody could depend on it.
Scope: product
The page had grown to roughly ten cards in one flow, and three more surfaces
were wanted on it: installed software, live metrics, and the service controls.
Stacking those into the same column would have buried what the page is for.
Tabs were mocked and rejected on 2026-08-28, on the grounds that the strip
duplicated the sidebar and cost hero space. That objection is answered by
where the strip sits: the hero and the triage band stay above it and never
move, so "is this machine OK" is still answered without a click, and the strip
partitions the detail underneath rather than navigating anywhere.
WHY A PANEL DECLARES ITS OWN GROUP
Most cards on a detail page are not the page's. They arrive through the
ADR-010 get_asset_panels hook, so a page that sorted panels into tabs by
knowing their ids would defeat the hook it is built on. Panels now carry an
optional group instead, and the page only knows the five group names.
Resolution happens server-side, in the asset-panels route, so that an absent
or misspelled group becomes 'details' before the client ever sees it. The
client renders known groups only; an unrecognised value that survived to it
would be a panel that exists and is displayed nowhere. Contract 0.28.0 ->
0.29.0, additive - every existing plugin keeps working untouched, and the
endpoints status panel is deliberately left ungrouped so the default path is
exercised by a shipped plugin and not only by tests.
A tab is offered only when something fills it. An empty tab reads as broken
rather than as "this site does not collect that", so the strip is built from
the groups that actually returned data, and a tab that disappears on refetch
falls back to Details rather than leaving the page blank.
AssetDetailTabs is core, not a PCDetail component: machines, printers,
network devices and measuring tools all have the same shape, and the shared
asset-detail shell they need starts here.
Scope: product
A link is built by formatting that protocol's own linktemplate, and the
templates are rows in accessprotocols that a site can edit. So no scheme
is a fixed fact about the product, and writing one down as though it were
misleads whoever reads it next.
Two places did. The guest-grant docstring said the detail route resolves
"working rdp:// and vnc:// links", which is wrong for the row we actually
ship: VNC is com.realvnc.vncviewer.connect://{host}:{port}, the RealVNC
handler, not vnc://. And the protocol admin form offered vnc://{host}:{port}
as its placeholder, in the one screen where somebody authors a new template,
so the example contradicted the seeded data it sits next to.
Neither changes behaviour. _computer_access_links already reads the
template from the database and the frontend only ever renders the resolved
link, so nothing in the code assumed a scheme - only the prose did.
Scope: product
The sidebar box was drawn for anyone signed in, on the reasoning that
offering it to a signed-out reader gives them a control that can only
answer 401. That stopped being the right test when the guest role became
able to hold assets.view: the box was hidden from exactly the visitor the
grant was reopened for, and a guest could reach search only by typing
/search?q= by hand.
The client cannot decide this. Its hasPermission getter reads the signed-in
user's permission list and a guest has no user at all, so asking it about a
guest answers no for everyone. get_navigation already had the problem and
already solved it by letting the server decide, so capabilities does the
same thing and reports what the CALLER holds.
It reports booleans about the caller and never what the guest role holds in
general, so it tells an anonymous reader nothing the rendered page does not.
Public because it is the pre-login call that decides whether the box is
drawn at all.
Members who lack assets.view now lose a box that only ever 401'd for them.
On a failed call the box stays hidden rather than falling back to the old
signed-in test, which would put it back in front of those same members.
Scope: product
Inverting the grantable rule from the .view suffix to explicit declaration
dropped every permission that had qualified by how it was spelled. Two of
them were shipped guest features with grants already in the field.
guides.view: the nav item and list_guides both reason in comments about a
signed-out reader holding it, and the route is written around one - drafts
are filtered out, search matches the published revision so draft wording
cannot leak through a hit, and the author branches cannot open for a caller
with no user id.
tools.view: without it a guest holding tools.secrets has permission to use
Secure Send and no way to reach the page, because tools.view is what puts
the entry in the nav. The tools themselves run in the browser and touch no
data.
The failure was worse than a missing checkbox. update_role refuses the
WHOLE request when any submitted permission is undeclared, and the role
grid resubmits what the role already holds, so a site that granted either
of these under the old rule could no longer save the guest role at all -
including to tick assets.view, which is the grant the shop floor needs to
look up an asset number without signing in.
The test asserts both stay grantable, so narrowing the set again means
looking at what is already granted in the field rather than only at what
the catalogue declares.
Scope: product
Global search is gated on assets.view, and that was the whole check.
Behind it the route queried twelve domains: applications, the knowledge
base, guides, the staff directory, assets, measuring tools, USB devices,
printed parts, custom fields, notifications, hostnames, and vendor and
model reference data, plus addresses and subnets for an IP-shaped query.
None of them consulted their own permission.
So anyone who could look up an asset number could also read the staff
directory out of the HR database, the USB checkout history and who holds
which drive, and the network inventory. The SEARCH_DOMAINS settings look
like they bound this but they are site feature toggles, not
authorization.
Each domain now answers only to a caller who holds its permission.
Assets, custom fields and vendor/model stay on assets.view because all
three emit asset rows routed to asset pages.
caller_holds rather than require_permission: refusing the whole request
because one domain is off-limits would make search useless to anyone
holding less than everything. A domain the caller cannot see contributes
nothing and the rest still answers.
This is what makes assets.view safe to grant the guest role again. It
left the grantable set in 3474028 because of the leak above, and that
took the shop floor's asset lookup with it - a signed-out visitor at a
machine could no longer search for the asset number in front of them.
Now the grant reaches assets and nothing else.
The first version of the leak test passed with the gates removed. The
staff directory is an external HR database and is unreachable under
test, so the domain silently returned nothing and the assertion proved
nothing. It patches _employee_rows now, and fails when the gate goes.
Scope: product
(cherry picked from commit 0a01ddcf25)
Rows sorted on iscritical then on printer NAME. Critical is everything
at or below 5%, so a cartridge at 5% and one at 0% tied on the first key
and the alphabet decided which came first.
Wrong order on its own, and worse than it looks: the dashboard queue
keeps only the first few rows per source, so a printer that was
genuinely empty was truncated off the card while a fuller one held its
place. Reported from the floor as "starts at 5% and hides one at 0%",
which is exactly the two bugs compounding.
remaining is already normalised for waste cartridges - a full waste bin
reads as low remaining - so one ascending sort is worst-first for every
supply type, and iscritical falls out of it rather than needing its own
key. Name only breaks ties.
worstremaining is carried on the row because the supplies list is
rendered to text ("Black 0%") before it reaches the client, so the
number the sort needs is gone by then.
The toner forecast report was already worst-first (it sorts on daysleft)
and is unchanged.
Scope: product
(cherry picked from commit 6644a07ae1)
The export rsyncs the WORKING TREE, so .gitignore does not apply to it:
anything ignored-but-present reaches the publication repo unless it is
named as an exclude. devnotes/ is declared never-committed and holds
review writeups and chat handoffs, and it went straight through.
The scrub gate stopped it, which is what that gate is for, and nothing
was committed or bundled. But a gate failure is a stop, not a filter,
and the fix a hurried person reaches for is deleting their notes to get
a bundle out.
Same shape as the stray .env.bak that reached this tree once before, so
the rsync now carries a note saying why excludes live there rather than
in .gitignore.
Scope: product
(cherry picked from commit f1136d920d)
Delivery descriptors for run_script and update_app are resolved when the
agent claims a command, not when an operator queues it, and the command
row recorded no digest: the enqueue path resolved the descriptor purely
to validate it and threw the result away.
So what an operator approved and what an endpoint runs were two separate
lookups against mutable content, minutes or hours apart. Anyone able to
replace a script body or an installer package could do so in the gap.
The claim then re-resolves, the descriptor faithfully describes the new
bytes, the agent verifies correctly, and it runs as LocalSystem. The
audit trail names the operator who queued the original.
The gate on the verb and the gate on the payload were also different
grants: pushing needs endpoints.deploy, but replacing what gets pushed
needs only applications.edit.
pinnedsha256 records the digest as it stood at enqueue, and the claim
refuses a descriptor that has moved, failing the push with an audit row
instead of shipping it. Null for actions that deliver no content, and
for rows queued before this migration, which claim as before. A
descriptor that vanished entirely is not treated as tampering: that path
already sends the command through undescribed so the agent fails closed.
Separately, a result may now only finalize a command that is actually
running. That needs no agent change and closes two things: reporting on
a command that was never claimed, and overwriting a recorded failure
with a success.
The agent now sends its hostname on the result callback so the server
can check it owns the command it is finalizing. The shared collector key
does not say who is calling, so without it any agent can post a result
for any command id. The server treats the parameter as optional for
agents that predate this and enforces it when present, so there is no
flag day. This is a floor, not a fix: only a per-endpoint credential
makes the caller's identity provable, and that is tracked separately.
The .NET side is unbuilt here and needs a build plus a win11 VM run.
Scope: product
(cherry picked from commit d64b7122ab)
client_ip() read the first X-Forwarded-For hop unconditionally. The
login rate limit and the account lockout are keyed on that value and the
audit log records it, so rotating the header per request bought an
unbounded budget and an untraceable trail.
This walked around TRUSTED_PROXY_COUNT, which exists to decide exactly
this. The app leaves ProxyFix off by default and says why, ten lines of
comment away: ProxyFix trusts X-Forwarded-*, a client can send those
itself, and the login limiter is keyed on the address. Then client_ip
read the header anyway, so the stated protection never applied.
Resolution is now from remote_addr, which ProxyFix has already filled
from the header wherever a proxy is genuinely configured. Both cases end
at the same trustworthy source, and the port still comes off because ARR
forwards clientip:port and ProxyFix passes it through verbatim.
geenforce and tools had each hand-rolled the same mistake for their own
limiters. Rather than duplicate security-relevant logic a third time,
client_ip and strip_port join the shopdb.api surface (contract 0.28.0).
geenforce's _trusted_client_ip, which already refused the raw header for
its IP allowlist, is untouched and was the model for this.
The stability table was also missing rows for 0.26.0 and 0.27.0.
INSTALL-WINDOWS-IIS.md never mentioned TRUSTED_PROXY_COUNT, so a site
installed from it runs behind IIS with the setting at 0: every request
looks like it came from IIS, the whole site shares one rate-limit
bucket, and the audit log records the proxy for every action. It now
says so, and says not to set it when nothing is in front.
Three test modules simulated separate callers by setting
X-Forwarded-For, which is the header a real attacker rotates to escape
the limit. One of them, test_client_ip_prefers_forwarded_first_hop_
stripped, asserted the defect outright. The suite could not have caught
this. They now drive REMOTE_ADDR, with a regression guard that forging
the header buys nothing.
Scope: product
(cherry picked from commit 7260244b7e)
There is no token denylist in this product, so the JWT user loader is
the only place a deactivated account can be stopped, and it did a bare
lookup by id. Somebody deactivated at 09:00 kept a working access token
until it expired, and a refresh token that minted fresh ones for thirty
days.
/api/auth/refresh and the API-token path already checked isactive. This
makes the loader consistent with them rather than inventing policy.
Scope: product
(cherry picked from commit 9c1848fc61)
guest_grantable_permissions() opened any permission whose name ended
'.view' or '.read' to the guest role, on the reasoning that reading is
the safe default. That is true of a permission gating a list of asset
numbers and false of one gating bytes, and a suffix cannot tell them
apart. Twenty-four of eighty-five qualified and none had been decided:
applications.view reached the SYSTEM-context PowerShell bodies and
the installer binaries pushed to bays
backups.view returned the full payload that the separate
backups.download permission exists to protect
assets.view gated a global search that queries the staff
directory, USB history and network inventory with
no per-domain check of its own
audit.view usernames, addresses, the before/after of every
settings.view change, and every setting row - both admin
category, opened by naming alone
users.view gates nothing at all; list_users is
require_role('admin')
Every entry is now declared, by CORE_GUEST_GRANTABLE or by the plugin
that owns the permission, having read what its routes actually return.
Adding one is a code change, which is the point: it puts the decision
in a diff rather than in a suffix.
The two routes serving script bodies and installer packages move to a
new applications.download permission, which is not grantable, so a site
can still open the catalogue metadata to its floor without the bytes
going with it.
The declared set preserves the grants this site had already chosen:
applications.view, computers.view, endpoints.view, reports.view,
tools.secrets, warranty.view.
Two known gaps are recorded on the hooks that expose them rather than
silently carried: the computer serializer publishes every column plus
resolved rdp:// and vnc:// links, and the per-asset endpoint panels are
keyed on a small integer so walking ids reconstructs much of the roster
that endpoints.fleet exists to protect. Neither is widened here.
A shipped test was granting guests audit.view and passing.
Scope: product
(cherry picked from commit 347402896c)
A multi-agent review of the merged branch produced 29 findings that survived
adversarial verification. This clears the blockers and the four worth fixing
before it reaches main. The design held up well: path-traversal defence on all
four serving routes is clean, the agent's SHA-256 check fails closed, enqueue
authority is proportionate and audited, and no site literals or credential leaks
in the new C#.
THE TESTS NEVER RAN
pytest.ini sets testpaths = tests, so plugins/endpoints/tests/ was collected ZERO
times. Fifty-five lines of delivery assertions were dead code, and run directly
five of them failed. plugins/printedparts/tests/ had been invisible for longer.
That is the worst shape a gap can take: the suite is green, the coverage is
imaginary, and the collector auth on a download route could be deleted without
anything going red.
Both files moved under tests/. tests/test_no_hidden_tests.py now fails if a test
file appears anywhere pytest will not look; it caught a staged copy inside the
installer bundle on its first run, and was verified by planting a violation.
The five failures were stale expectations, and they split by cause. His: the
applications panel made six panels where his test said five, and 'applications'
belongs in KNOWN_PAYLOAD_KEYS now that it is persisted - his own test contradicted
his own feature. Ours: we renamed the events card to activity, moved the
fleet-wide cards to endpoints.fleet, and made the list route a bare route('')
which Flask rejects unless the blueprint is mounted under a prefix.
FOUR FIXES
The snapshot table's key was a 32-bit autoincrement on a table rewritten
wholesale every check-in, so it climbs at (endpoints x apps) per report and
exhausts on MySQL. Widened while the table is unreleased. Widening it broke every
insert on SQLite, which only autoincrements a column typed exactly INTEGER
PRIMARY KEY, so it carries a dialect variant - and nothing caught either fault,
because no test inserted into that table at all. One does now.
Ingest truncated displayname but not displayversion or publisher. On strict MySQL
an over-long registry value fails the whole check-in transaction, so one machine
with a malformed Uninstall entry goes silent entirely. SQLite does not enforce
lengths, so the suite would never have shown it.
update_app's version: branch pinned AppVersion.sha256, but that route resolves the
version to its APPLICATION and serves application.installpath - so the digest
described an artifact the route never sends, and where it was null the agent had
nothing to verify and failed open. It now pins the file actually served.
The same branch ended the delivery URL at the id, so an MSI arrived with no
extension and could not be routed through msiexec. It carries the filename now.
Agent packaging moved off require_role('admin') to a new endpoints.agentpackage
permission. The generated ZIP carries a live collector key, so this is a heavier
grant than deploying to one endpoint - and a role cannot be given to one person
for one job the way a permission can.
Full suite 2231 passed (2198 before, plus the 22 that never ran and 11 new),
frontend 314.
Commit 0c2b816 added Script to shopdb.api so a plugin can read the ShopDB-hosted script library that backs run_script delivery, but left __contract_version__ at 0.26.0. This bumps it to 0.27.0 so a plugin can pin core_version against the widened surface. Additive name, minor bump, matching the existing convention. The docs/PLUGIN-HOOKS wiki page (not tracked in-repo) still needs the Script entry.
Reverses everything the installer and site-agent generator create: stops and deletes the service, kills agent/tray processes, removes the tray Run key and the HKLM\SOFTWARE\GE\ShopDB credential hive in both registry views, drops the Defender path exclusion, and deletes the install directory and ProgramData tree. Every step is best-effort and idempotent so it is safe to re-run and safe on a partially installed host.
Terminate TLS in gunicorn using a host-generated cert mounted read-only under ./certs, so the browser sees a secure context. Secure-context-only web APIs (navigator.clipboard and others) refuse to run over plain http on a non-localhost origin, which the FQDN-accessed test box hits. Healthcheck now probes https with -k to match the TLS listener. A site swaps in its own cert or fronts this with its own IIS/nginx proxy.
Bumps the agent to 1.4.1. run_script fetches a ShopDB-hosted script by digest and runs it in the script's configured context (LocalSystem or the interactive logged-on user), replacing the local scriptpaths allowlist; user-context runs wait for exit and report the real exit code instead of fire-and-forget. update_app resolves an app-level or version-level install source, fetches the pinned bytes, verifies sha256, and runs silently. Both stage into a fixed Defender-excludable folder (C:\ProgramData\ShopDB\EndpointAgent) instead of the LocalSystem temp path. Adds the DeliveryClaim contract for the claim payload.
Adds the two delivery actions to the endpoints command allowlist, each gated on a new endpoints.deploy permission ON TOP OF endpoints.command so pushing software is a heavier grant than a routine toggle. Enqueue validates update_app appversionid ownership and installer presence before queueing. New collector-authenticated download routes let the agent fetch installer packages and script bodies by digest rather than pre-staging them or sending bodies over the command wire. Wires the scriptsApi client, the siteagent generator routes, and a temporary dev-only Delivery test bench (view/route/nav) for exercising the path while the production UI is built.
AppVersion gains sha256 and silentargs so the endpoint update_app push can pin exact installer bytes and carry per-version silent switches; Application gains installersha256 for the app-level install source. New ShopDB-hosted script library (Script model plus scripts API) backs run_script delivery, with the body fetched by digest on push rather than pre-staged or sent over the command wire. Two idempotent migrations with real downgrades. Script added to the shopdb.api contract surface for plugin use.
The command queue is at-most-one-in-flight, but nothing ever expired a stuck command. When an agent claimed a command (RUNNING) then died before reporting a result - killed, uninstalled, or offline mid-flight - the row stayed in-flight forever and every operator action for that endpoint returned 409 'already in flight'. Reap in-flight rows past a staleness ceiling (15m for RUNNING, 60m for PENDING) to a new STATUS_EXPIRED at enqueue time, inside the same transaction, so the queue self-heals lazily the next time someone queues work. No background job and no schema migration - status is free-text.
A misconfigured BaseUrl (plaintext http:// against the TLS port) made the command long-poll fail at connect in milliseconds. The worker misread that fast failure as a normal idle hold-window close and re-polled with no delay, spinning into a request storm. Gate the idle-close path on the poll having stayed open at least MinIdlePollDuration, so a sub-second transport failure takes the transient backoff instead; add a MinPollInterval floor so no path can exceed a safe request rate. Also reject a non-loopback http:// BaseUrl at startup with a single clear error, turning a silent runtime flood into a fast fail. Loopback http stays valid for local dev.
Several days of endpoint-agent work left DB dumps, sample datasets, captured command output, generated maps, vuln scan output, merge-kit exports, and staged agent publish trees in the tree. None are product source. Ignore them so status shows only real work and they cannot be committed by accident. exports/ stays on disk for reference. agent/.gitignore also reserves dev-bin/ and _stage/ for delivery-build scratch.
Tracking docs/openapi.json (so the app can serve the spec at runtime) makes docs/ exist on the published GitHub mirror, which defeated the skipif(not DOCS.is_dir()) guard in the docs publish/version tests: they then ran against a prose-less checkout and failed on CI. Key the skip off docs/*.md prose instead, matching the canonical published-checkout definition in scripts/check-gates-ran.sh.
Also: the OpenAPI generator emitted a trailing-slash path (/api/docs/), which a strict proxy like IIS treats as a redirecting resource a subpath-mounted client cannot follow; normalise trailing slashes in _norm_path and correct the committed spec. And allow docx (python-docx) as an undeclared import: it is used only by scripts/docs/generate_docx.py, a developer tool that never runs on a deployed site.
The endpoint agent already reports an installed-software list (SBOM), but ingress dropped it: the collector schema had no entry for it and there was nowhere to store it, so operators could not see what is installed on a PC.
Persist it as a per-asset snapshot in a new endpointapplications table, replaced on every report (current inventory is what operators ask for, not install history). Add applications to the collector schema and KNOWN_PAYLOAD_KEYS so ingress stops discarding it, a read route for the raw collection, and a collapsed-by-default Installed software panel to surface it. appid is nullable groundwork for a future Managed Applications link; no matcher is wired yet.
Standalone one-shot enrollment tool that creates a ShopDB PC + Endpoint entry using only the endpoints collector key, reusing the existing collector adopt path. Collects identity, WMI hardware, and installed-software SBOM; requires admin. Server-side packaging route attaches to the existing endpoints blueprint; no core contract change.
The agent now reports installed applications from both Uninstall registry hives (64-bit and WOW6432Node) as {displayname, displayversion, publisher}, mirroring the Assess-Orchestrator applications collection. Registry-only, no Win32_Product/WMIC self-repair and no AppX. The server logs it as an unpersisted key until the SBOM store is built.
The endpoint agent now sends disk, bios, memory, processor, osinfo, and domain telemetry. The server does not persist these yet, so ingest logs any unpersisted payload keys with a compact preview. This makes incoming telemetry observable in the logs before any UI or schema work, so we can confirm the agent is sending it and enhance the UI later.
wj held nothing main lacked - git cherry marks every one of its commits as
already present - but promotion cherry-picks, so the histories diverged and a
fast-forward is not possible. Merging rather than resetting keeps the branch
recoverable and needs no force-push.
Levelling first so wilm's patches apply against current product code.
# Conflicts:
# CONTRIBUTING.md
# frontend/src/views/Dashboard.vue
# tools/wj-status.sh
Asked twice what "All assets" covered, which is the signal a label is
underspecified rather than a reader being slow.
The endpoint already counted every status to build its rollup and then returned
two of them, so a reader could see In Use and had no way to learn where the rest
of the total went. It now returns them all, and the panel lists them under the
total - indented and quieter, because they break the total down rather than
adding to it.
Three things this makes visible that were not guessable:
Retired assets are IN the total. Retired is a status on an active record, not
the same as deactivated, and seven of them sit inside this site's 610.
There is an "Inventory" status. I would have hardcoded In Use, In Repair and
Retired and silently dropped it. The rows are built from whatever the API
returns, sorted by count, so a status a site adds appears rather than
disappearing - which matters for sites adopting with their own seed data.
Rows held back by isactive are in NO status bucket, so they cannot be found by
adding the buckets up. That gap is exactly what makes a dashboard total
disagree with a list page, so the count is reported and captioned.
The caption states the counting policy - active records only, dual-bay pairs
counted once - and the dual-path clause renders only when that site toggle is
on, since asserting it unconditionally would be false at a site that has it off.
Full suite 2182, frontend 314.
The panel had no group label before it was one panel, so grouping it needed a
heading and it got 'Estate' - RMM idiom, and the word the leadership documents
already used for the whole of what a site owns.
Wrong register for this surface. The sidebar already groups exactly these things
under a section named assets, so the dashboard had invented a second word for a
group the app already names, and the panel's own total row says 'All assets' -
the heading and the total disagreed inside one panel. 'Estate' is also
enterprise-IT and largely British idiom; it earns its place in an executive
summary and not on a screen read at 6am.
The scoped classes follow the label, so the markup does not keep a word the UI
stopped using. The documents keep 'estate', where it is doing real work for a
leadership reader.
Scope: product
The bundle became one file per branch, but the frontend dists did not: they are
still frontend-dist-subpath-{ops,shopdb}, fixed paths, and update-prod-server.cmd
robocopies the shopdb one onto the production server with /MIR - a full mirror -
with no idea which branch built it.
So --dist from a site branch would have replaced production's frontend with
unreleased work at the next deploy, silently, and nothing downstream could have
told. Worse than the bundle collision it was sitting beside, because the dist is
what actually lands on the server.
Branch-qualifying the directories would only move the trap to whoever pointed a
deploy at the wrong one. Production runs the product, so the export now refuses
and says which commits to promote instead.
Scope: product
Promotion cherry-picks, which gives the commit a new sha on main. wj-status
compared by sha via main..wj, so after promoting ten commits it still listed all
ten as unpromoted - and would have invited promoting the same work twice.
git cherry compares patch content instead and marks an already-present commit
with a leading '-'. When nothing is genuinely outstanding it now says so, counts
what has already landed, and shows how far the branch is BEHIND main with the
rebase to run. Behind was the state that mattered here and the tool could not
express it at all.
Scope: product
Recorded that wilm no longer existed, on the evidence of the local gitea clone
where it was merged and its ref deleted. That was wrong. It lives on the
corporate GitHub and the other site is developing the .NET endpoint agent on it.
The dev box cannot see that branch - no account there - so its absence locally
was never evidence of anything, and fetch-wilm-branch.cmd on the share exists
precisely to bring it back for review. Says so now, so the next reader does not
draw the same conclusion from the same missing ref.
Scope: product
The work-PC instructions began with 'git checkout <branch>', which fails the
first time a branch goes over because it does not exist in that clone yet -
exactly the moment somebody follows them most carefully. Nothing to check out,
and nothing to merge into either.
Now prints both: fetch then checkout -b then push -u the first time, and the
checkout/fetch/merge/push cycle afterwards.
Scope: product
(cherry picked from commit 1fe788a3ed)
promote-to-main.sh checks out main. The tooling lives only on the site branch,
so that checkout removes tools/promote-to-main.sh while the script is still
running - and bash reads a script lazily, by file offset, so it would carry on
executing whatever bytes happen to land there. While manipulating branches.
Not theoretical: main does not have tools/ right now, so the first real use
would have hit it.
Both versions now copy themselves to a temp file and re-exec from there. The
bash copy unlinks itself immediately, since the open handle keeps working. The
Windows copy has the same hazard for the same reason - cmd.exe re-seeks the
batch file after every command - and two extra traps worth naming: the copy runs
from %TEMP%, so the repository root has to travel with it rather than being
derived from %~dp0, and %errorlevel% inside a parenthesised block is expanded
when the block is PARSED, before the call it is meant to report on.
Scope: product
(cherry picked from commit 6d42110a91)
A guest got a heading and nothing else.
THE STRUCTURAL FAULT WAS MINE
Moving the Estate panel into DashboardCards' rail slot put it behind that
component's root v-if, which was `liveCards.length > 0`. A guest is permitted
zero cards, so the whole component disappeared and took the slot with it. Before
that move the tiles were top-level and always rendered. The wrapper no longer
vanishes with the cards.
TWO LIES THE EMPTY STATE WAS TELLING
"Nothing needs attention" is now shown only to somebody who can see the checks.
A reader permitted none of them is not looking at a healthy fleet, they are
looking at nothing, and saying otherwise is the same fault as reporting Low
Supplies 0 while the monitoring is down. The count of PERMITTED cards is now
tracked separately from the count that currently hold rows.
The Estate panel no longer renders zeros for a caller who cannot read totals.
/api/dashboard/summary needs assets.view, and the failure was swallowed into an
empty object that drew a tidy column of zeros - an estate of nothing, stated
with confidence.
WHAT A GUEST GETS INSTEAD
Where they can go: the same permission-filtered navigation the sidebar uses,
which the server already filters to the guest role for a signed-out visitor. So
it cannot offer a link that dead-ends at the router guard, and this page names no
plugin it had to know about. Verified against the running instance - a guest is
offered Guides and Tech Tools, matching guides.view and tools.view.
ALSO: the measuring tools row
totalassets counts five types - machines, PCs, network, printers AND measuring
tools - but only four had rows, so the total stopped matching the addends above
it wherever gages exist: 610 of visible rows under a stated 729. That is exactly
the fault this panel was built to remove, and it looked right here only because
this database has no measuring tools. Every counted type now has a row.
Frontend 314 passed.
(cherry picked from commit 0f2da9beaf)
The export wrote a single shopdb-flask-pub.bundle whatever it published. That
was fine while only main was ever published, and became a trap the moment it was
not: exporting a site branch silently overwrote the bundle holding main's work,
and the work PC's documented "git fetch <bundle> main" then failed because that
ref was no longer in the file.
It had already happened here - two wj exports overwrote a main bundle from the
same morning. The failure is at least loud on the work PC rather than pushing
the wrong branch, but the main bundle has to be regenerated.
Now shopdb-flask-pub-<branch>.bundle. Two branches cannot share one filename.
Scope: product
(cherry picked from commit 1311dea113)
The branch checkout landed after the rsync, where git checkout --force reverts
the tree to the branch's committed state and discards everything rsync had just
written. The export then found no changes and published nothing, while still
printing a bundle path and looking like it had worked. A silent no-op is the
worst shape for this: the next person believes the export ran.
Scope: product
(cherry picked from commit 06c584240e)
export-github.sh syncs the WORKING TREE, not a branch, and then committed
whatever it found onto the publication repo's main and printed
"git push origin main".
So running it while checked out on a site branch published that site's work AS
THE PRODUCT, to the repository every other site consumes, and said nothing. The
branch model made that reachable in one wrong command, from a checkout that
looks completely normal.
It now reads the current branch, mirrors it in the publication repo (creating it
from main the first time, as in the dev repo), bundles that branch, and prints
work-PC instructions naming it. Publishing anything other than main also prints
a reminder that other sites read main and these commits are not there yet.
The publication checkout uses --force, which is safe only because that tree is a
mirror: the rsync has already replaced it, so there is nothing local to lose. It
would not be safe anywhere else.
Also cleans up the publication repo: origin pointed at a bundle in a scratchpad
from a session that ended long ago, and two orphaned remote-tracking refs
(github/main, wilm/review-fixes) survived the removal of their remotes. None of
it broke the export, which never pushes, but a `git fetch` in there failed with
a confusing path.
Scope: product
(cherry picked from commit 81df9607e7)
The convention existed only in the scripts' comments and in one conversation,
which is not where somebody joining looks. CONTRIBUTING is.
Records the two branches, the Scope trailer, the promote flow, and the point
that most work is product: ADR-015 puts site differences in settings, seed data
or scripts/site_imports/<site>/, so a site branch is about timing rather than
locality.
Also names the failure worth watching. Promoting something site-specific by
accident is caught by the ADR-015 gate on the way into main. The real risk is
product work stranded on a site branch because nobody decided, which is what
turned wilm from a merge into a port.
Scope: product
(cherry picked from commit 1d4ac11572)
The scripts could move a commit to main but nothing said which commits should
go, and that decision does not survive being deferred: months later it is a list
of subject lines and a guess.
So it is recorded at commit time, as a trailer:
Scope: product the rest of the world should have it
Scope: site this site alone, keep it here
tools/wj-status.sh reads them and prints main..wj in three groups, with the
promote command for the ones marked product.
The path fallback is deliberately weak. Under ADR-015 the only site-specific
home in the tree is scripts/site_imports/<site>/ - everything else is meant to
be product, with settings and seed data carrying what differs between sites. So
"touches only site_imports" is the one confident site verdict paths can give,
and everything else is reported UNMARKED rather than guessed into a bucket. A
tool that quietly sorted commits by directory would be wrong often enough to
stop being read.
The unmarked message says which way to lean, because the failure mode here is
not promoting something site-specific by accident - the ADR-015 gate catches
that on the way in. It is product work sitting on wj forever because nobody
decided, which is how the wilm branch became a port rather than a merge.
Scope: product
(cherry picked from commit 0400a58786)
Same two operations as the bash pair, for driving git from Windows rather than
from the development box.
CREDENTIALS ARE HANDLED DIFFERENTLY, ON PURPOSE
The bash version reads a token out of secrets.md and feeds it to git through
GIT_ASKPASS, because that box has no credential store. Windows has one: Git for
Windows installs Git Credential Manager and enables it by default, so `git push`
prompts once and Windows Credential Manager keeps it.
So the token handling is deliberately NOT ported. Copying a token into a script,
an environment variable or a remote URL on a machine that already has a
credential store is strictly worse than the thing it would replace, and the
comment in the file says so, because "make it match the other script" is exactly
the tidying somebody would otherwise do later.
THE PROMOTE SCRIPT REFUSES TO RUN WITHOUT BASH
scripts/check-naming-and-style.sh is a shell script, and it carries the ADR-015
site-literal check - the one gate that makes a site branch survivable rather
than a slow leak into the product. Skipping it on a machine without bash would
defeat the entire point of the script, so it stops and says how to fix it
instead of promoting without it.
CRLF
.gitattributes already pinned *.sh to LF so Git Bash can run them; *.cmd is now
pinned to CRLF for the same class of reason in the other direction. cmd.exe
parses a lone LF inconsistently around labels and goto, so an LF-only batch file
can run its happy path and then jump nowhere on the error path - which in a
promote script is the path that matters.
(cherry picked from commit 49a898fc5a)
wj holds this site's work; main is the product every other site sees and the
publication mirror exports. Two scripts, one for each direction.
tools/git-push.sh pushes a branch without leaving the token behind. The way this
was being done by hand - rewrite remote.origin.url to embed the token, push,
rewrite it back - writes the token into .git/config in plaintext on every push,
and leaves it there if the push fails or the terminal closes mid-way, where
`git remote -v` will happily print it. Putting it in the URL on the command line
is no better: argv is visible in `ps` to every user on the box. The token now
goes in through GIT_ASKPASS, so it never reaches the config or the process list
and there is nothing to clean up when the script dies.
tools/promote-to-main.sh cherry-picks NAMED commits onto main, rather than
merging the branch. Merging takes the site-specific work along with the product
work, which is how a site branch quietly becomes the product; naming the commits
keeps that decision explicit and per change.
It runs main's gates after the pick and before any push, and does not push at
all - it prints what landed and leaves that to a human. The ADR-015 site-literal
check is the one that earns its place: it catches a hostname or share path
riding along in a commit that looked generic, which is the failure that makes a
site branch dangerous rather than merely awkward. It caught this very commit,
which named the site in a comment.
Both refuse a dirty tree. Pushing one pushes the last commit rather than what is
in front of you, and cherry-picking from one mixes uncommitted work into the
promotion. On failure the promote script aborts the pick and returns you to the
branch you started on, with main untouched.
(cherry picked from commit 6d9954c6f9)
The work-PC instructions began with 'git checkout <branch>', which fails the
first time a branch goes over because it does not exist in that clone yet -
exactly the moment somebody follows them most carefully. Nothing to check out,
and nothing to merge into either.
Now prints both: fetch then checkout -b then push -u the first time, and the
checkout/fetch/merge/push cycle afterwards.
Scope: product
promote-to-main.sh checks out main. The tooling lives only on the site branch,
so that checkout removes tools/promote-to-main.sh while the script is still
running - and bash reads a script lazily, by file offset, so it would carry on
executing whatever bytes happen to land there. While manipulating branches.
Not theoretical: main does not have tools/ right now, so the first real use
would have hit it.
Both versions now copy themselves to a temp file and re-exec from there. The
bash copy unlinks itself immediately, since the open handle keeps working. The
Windows copy has the same hazard for the same reason - cmd.exe re-seeks the
batch file after every command - and two extra traps worth naming: the copy runs
from %TEMP%, so the repository root has to travel with it rather than being
derived from %~dp0, and %errorlevel% inside a parenthesised block is expanded
when the block is PARSED, before the call it is meant to report on.
Scope: product
A guest got a heading and nothing else.
THE STRUCTURAL FAULT WAS MINE
Moving the Estate panel into DashboardCards' rail slot put it behind that
component's root v-if, which was `liveCards.length > 0`. A guest is permitted
zero cards, so the whole component disappeared and took the slot with it. Before
that move the tiles were top-level and always rendered. The wrapper no longer
vanishes with the cards.
TWO LIES THE EMPTY STATE WAS TELLING
"Nothing needs attention" is now shown only to somebody who can see the checks.
A reader permitted none of them is not looking at a healthy fleet, they are
looking at nothing, and saying otherwise is the same fault as reporting Low
Supplies 0 while the monitoring is down. The count of PERMITTED cards is now
tracked separately from the count that currently hold rows.
The Estate panel no longer renders zeros for a caller who cannot read totals.
/api/dashboard/summary needs assets.view, and the failure was swallowed into an
empty object that drew a tidy column of zeros - an estate of nothing, stated
with confidence.
WHAT A GUEST GETS INSTEAD
Where they can go: the same permission-filtered navigation the sidebar uses,
which the server already filters to the guest role for a signed-out visitor. So
it cannot offer a link that dead-ends at the router guard, and this page names no
plugin it had to know about. Verified against the running instance - a guest is
offered Guides and Tech Tools, matching guides.view and tools.view.
ALSO: the measuring tools row
totalassets counts five types - machines, PCs, network, printers AND measuring
tools - but only four had rows, so the total stopped matching the addends above
it wherever gages exist: 610 of visible rows under a stated 729. That is exactly
the fault this panel was built to remove, and it looked right here only because
this database has no measuring tools. Every counted type now has a row.
Frontend 314 passed.
The export wrote a single shopdb-flask-pub.bundle whatever it published. That
was fine while only main was ever published, and became a trap the moment it was
not: exporting a site branch silently overwrote the bundle holding main's work,
and the work PC's documented "git fetch <bundle> main" then failed because that
ref was no longer in the file.
It had already happened here - two wj exports overwrote a main bundle from the
same morning. The failure is at least loud on the work PC rather than pushing
the wrong branch, but the main bundle has to be regenerated.
Now shopdb-flask-pub-<branch>.bundle. Two branches cannot share one filename.
Scope: product
The branch checkout landed after the rsync, where git checkout --force reverts
the tree to the branch's committed state and discards everything rsync had just
written. The export then found no changes and published nothing, while still
printing a bundle path and looking like it had worked. A silent no-op is the
worst shape for this: the next person believes the export ran.
Scope: product
export-github.sh syncs the WORKING TREE, not a branch, and then committed
whatever it found onto the publication repo's main and printed
"git push origin main".
So running it while checked out on a site branch published that site's work AS
THE PRODUCT, to the repository every other site consumes, and said nothing. The
branch model made that reachable in one wrong command, from a checkout that
looks completely normal.
It now reads the current branch, mirrors it in the publication repo (creating it
from main the first time, as in the dev repo), bundles that branch, and prints
work-PC instructions naming it. Publishing anything other than main also prints
a reminder that other sites read main and these commits are not there yet.
The publication checkout uses --force, which is safe only because that tree is a
mirror: the rsync has already replaced it, so there is nothing local to lose. It
would not be safe anywhere else.
Also cleans up the publication repo: origin pointed at a bundle in a scratchpad
from a session that ended long ago, and two orphaned remote-tracking refs
(github/main, wilm/review-fixes) survived the removal of their remotes. None of
it broke the export, which never pushes, but a `git fetch` in there failed with
a confusing path.
Scope: product
The convention existed only in the scripts' comments and in one conversation,
which is not where somebody joining looks. CONTRIBUTING is.
Records the two branches, the Scope trailer, the promote flow, and the point
that most work is product: ADR-015 puts site differences in settings, seed data
or scripts/site_imports/<site>/, so a site branch is about timing rather than
locality.
Also names the failure worth watching. Promoting something site-specific by
accident is caught by the ADR-015 gate on the way into main. The real risk is
product work stranded on a site branch because nobody decided, which is what
turned wilm from a merge into a port.
Scope: product
The scripts could move a commit to main but nothing said which commits should
go, and that decision does not survive being deferred: months later it is a list
of subject lines and a guess.
So it is recorded at commit time, as a trailer:
Scope: product the rest of the world should have it
Scope: site this site alone, keep it here
tools/wj-status.sh reads them and prints main..wj in three groups, with the
promote command for the ones marked product.
The path fallback is deliberately weak. Under ADR-015 the only site-specific
home in the tree is scripts/site_imports/<site>/ - everything else is meant to
be product, with settings and seed data carrying what differs between sites. So
"touches only site_imports" is the one confident site verdict paths can give,
and everything else is reported UNMARKED rather than guessed into a bucket. A
tool that quietly sorted commits by directory would be wrong often enough to
stop being read.
The unmarked message says which way to lean, because the failure mode here is
not promoting something site-specific by accident - the ADR-015 gate catches
that on the way in. It is product work sitting on wj forever because nobody
decided, which is how the wilm branch became a port rather than a merge.
Scope: product
Same two operations as the bash pair, for driving git from Windows rather than
from the development box.
CREDENTIALS ARE HANDLED DIFFERENTLY, ON PURPOSE
The bash version reads a token out of secrets.md and feeds it to git through
GIT_ASKPASS, because that box has no credential store. Windows has one: Git for
Windows installs Git Credential Manager and enables it by default, so `git push`
prompts once and Windows Credential Manager keeps it.
So the token handling is deliberately NOT ported. Copying a token into a script,
an environment variable or a remote URL on a machine that already has a
credential store is strictly worse than the thing it would replace, and the
comment in the file says so, because "make it match the other script" is exactly
the tidying somebody would otherwise do later.
THE PROMOTE SCRIPT REFUSES TO RUN WITHOUT BASH
scripts/check-naming-and-style.sh is a shell script, and it carries the ADR-015
site-literal check - the one gate that makes a site branch survivable rather
than a slow leak into the product. Skipping it on a machine without bash would
defeat the entire point of the script, so it stops and says how to fix it
instead of promoting without it.
CRLF
.gitattributes already pinned *.sh to LF so Git Bash can run them; *.cmd is now
pinned to CRLF for the same class of reason in the other direction. cmd.exe
parses a lone LF inconsistently around labels and goto, so an LF-only batch file
can run its happy path and then jump nowhere on the error path - which in a
promote script is the path that matters.
wj holds this site's work; main is the product every other site sees and the
publication mirror exports. Two scripts, one for each direction.
tools/git-push.sh pushes a branch without leaving the token behind. The way this
was being done by hand - rewrite remote.origin.url to embed the token, push,
rewrite it back - writes the token into .git/config in plaintext on every push,
and leaves it there if the push fails or the terminal closes mid-way, where
`git remote -v` will happily print it. Putting it in the URL on the command line
is no better: argv is visible in `ps` to every user on the box. The token now
goes in through GIT_ASKPASS, so it never reaches the config or the process list
and there is nothing to clean up when the script dies.
tools/promote-to-main.sh cherry-picks NAMED commits onto main, rather than
merging the branch. Merging takes the site-specific work along with the product
work, which is how a site branch quietly becomes the product; naming the commits
keeps that decision explicit and per change.
It runs main's gates after the pick and before any push, and does not push at
all - it prints what landed and leaves that to a human. The ADR-015 site-literal
check is the one that earns its place: it catches a hostname or share path
riding along in a commit that looked generic, which is the failure that makes a
site branch dangerous rather than merely awkward. It caught this very commit,
which named the site in a comment.
Both refuse a dirty tree. Pushing one pushes the last commit rather than what is
in front of you, and cherry-picking from one mixes uncommitted work into the
promotion. On failure the promote script aborts the pick and returns you to the
branch you started on, with main untouched.
Splitting "cannot see the cartridges" from "nothing is low" was right, but it
treated two unlike cases as one. isconfigured means enabled AND a URL AND a
token, so its absence means the site did not ask for supply monitoring at all -
and a permanent "Supply monitoring is not configured" line is a card reporting
nothing wrong every day, which is how a board teaches people to stop reading it.
It matters for sites adopting ShopDB without Zabbix, who would otherwise inherit
a warning about a feature they declined.
Not configured now returns a plain empty and the card hides. Only configured and
unanswerable reports itself, because that is something meant to work that is not
working.
Also withdraws an unfounded inference. A dev box has no route to Zabbix by
design, so "not reachable" there is the expected state and says nothing about
what production can see. The value of this change is not that it revealed a
production fault; it is that when production's Zabbix does go down, the board
will say so instead of displaying "Low Supplies 0".
Four tests pin the three cases apart: unreachable reports, unconfigured stays
quiet, and a real zero from a reachable Zabbix is not flagged as blindness. The
fourth holds the envelope's rows, since carrying a reason changed the response
shape. Verified they are not vacuous by restoring the original branch in place -
the unreachable test fails against it.
Full suite 2182 passed.
TWO CONTROLS, ONE BOARD
Severity answers "how bad". It could not answer "what kind", and on a fleet
where eight endpoints go quiet at once, being able to set those aside and look
at the four crashes is the difference between a board and a list.
Chips per contributing check, coloured by its severity, composing with the
severity rollup above them. Both narrow what is already loaded rather than
refetching, so no combination can disagree with the counts. Picking a severity
that excludes the active chip clears the chip: two controls lit above an empty
queue, neither obviously wrong, is worse than losing the narrower one.
A chip counts every row, not the rows the queue had room for. A chip reading 5
above a "3 more" line would be its own small lie.
"LOW SUPPLIES 0" WAS NOT TRUE
Asked what happened to toner replacement, and the answer was worse than the
layout question that prompted it.
_get_low_supplies_data returned the same empty result for three different
situations: Zabbix not configured, Zabbix unreachable, and every cartridge
full. The dashboard strip therefore displayed "Low Supplies 0 / Critical 0"
while the monitoring was dead - and this instance is exactly that case, with
zabbix_enabled true against a URL nothing answers on.
So the strip removed in the previous commit was not information. It was
reassurance, and it read identically whether the printers were fine or nobody
could see them. That is the failure this codebase already names elsewhere: an
empty card and a broken card must not look the same.
The endpoint now distinguishes the three, and returns an envelope rather than a
bare list so a card can say WHY it has no rows. cardVisible shows such a card
whatever its empty rule, because "I cannot tell" is not "nothing to report".
It renders as a notice line, not a queue row: there is no asset to act on, and
burying "the monitor is dead" among the things it failed to find would be the
same mistake in a new place.
Note what this does NOT fix: the Zabbix URL on this instance is a dev
placeholder. The board will now say so instead of implying the cartridges are
full, but somebody still has to point it at a real server - and it is worth
checking what production points at, since nothing would have shown there either.
Frontend 314 passed (13 new), full suite 2178. The response shape changed from
a list to an envelope; no test depended on the old shape.
The board was one card per source in a fixed grid, and both halves of that hurt.
Every source got the same small box whatever it had to say. Warranties showed 5
of 50 behind "and 45 more"; the crash card truncated at 4; a one-row card sat
beside a four-row card and left a hole. Nothing had room, which is the complaint
that started this: not enough space to relay the data.
And a box per source flattened priority. An enforcement failure and a warranty
notice looked equally urgent because each owned a card, while the largest type
on the page belonged to counts that are true every day and need nobody.
WHAT THE PAGE IS NOW
A severity rollup, one worst-first queue of everything that needs a person, and
a row of context beneath it.
The queue merges every critical and warning row and sorts them, so reading order
is work order. It is full width, which is where the room came from. Rows are
capped per source, because fifty expiring warranties must not bury two crashed
PCs, and what did not fit is named per source with a link to it - a bare "and 45
more" that leads nowhere is worse than saying nothing.
The rollup counts ROWS, not cards, and filters the board in place rather than
navigating away, so the counts and the rows cannot disagree.
Severity, position and permission were already declared per card and already
sorted on. The old layout threw that away; this only stops it.
WHAT WENT, AND WHY IT SHOULD NOT COME BACK
Four asset tiles plus an "All assets" tile: machines + PCs + printers + network
IS all assets, so the page showed a total beside its own addends at the same
weight. It is now a line under a rule, where a total belongs.
The printer strip. It carried the printer count a third time - the tile row and
"Total Printers" were already the same number - and "Online" equalled total, so
three tiles held one fact. A genuinely low cartridge already reaches the queue
through the printers-supplies card, which is the moment anyone wants it.
Three cards whose whole job was to display 0.
Recent Devices: the largest block on the page, listing five machines that were
all "In Use" with business unit "TBD". Recency is not a reason to look at
something.
CONTRACT 0.26.0: 'shortname'
Cards may declare a short label for their rows in the merged queue. Their
'title' is written as a heading - "Endpoints with crashes or shutdowns" - and as
a row badge it ran wider than the hostname it qualified, putting the badge
rather than the asset at the start of every line. CSS-truncating the title was
tried first and was worse: four consecutive rows reading "ENDPOINTS WITH CR..."
say nothing at all. Optional and defaults to the title, so a plugin that does
not declare one is unaffected. Additive field, minor bump.
ALSO: "silent 1 days"
A meta suffix is written for the plural case and the card cannot know the value.
One row saying it was a shrug; the merged queue puts every quiet endpoint on one
screen, so it said it five times in a column and read as broken. Now singularised
at exactly 1, on the first word, keeping what follows: "1 event (48h)", "silent 1
day". The test asserting ' ms' survives is what caught the first version turning
it into ' m'.
DashboardRow.vue is extracted because the queue and the context panels render
the same row, and the map tooltip, link fallbacks and chips are too easy to get
subtly different in a second copy.
Verified on screen against live data. Full suite 2178 passed, frontend 301.
The triage band now answers the common case above the fold - what an asset
serves, or what drives it - so the full edge list below is reference rather than
the first thing to read. On a machine showing two relationships, both already
appeared in the band and the banner, and the card was a third copy.
Collapsed by default, using the same caret, roles and keyboard handling as the
collapsible panels on the same page rather than a second, slightly different
one.
The card is NOT redundant with the band and is not removed. It is the only
surface with Add Relationship and the per-row delete, and it keeps each edge's
exact type where the band deduplicates by asset. The printer page is the proof:
four rows there - usesprinter and defaultprinter, for each of two machines -
collapse to two band chips, and which machine DEFAULTS to that printer is a
distinction the band deliberately drops. Fine for triage, wrong for management.
THE COUNT IS OF WHAT IS RENDERED, NOT OF EDGES
A collapsed card that just says "Relationships" cannot be told from an empty
one, so the header carries a count. The first version counted raw edges and was
wrong on the first machine it was shown: a symmetric Dualpath link is stored in
BOTH directions in the legacy shape, so it is two rows in outgoing+incoming and
one "Connected" entry on screen. The badge read 3 above a card that shows 2 when
expanded - exactly the mismatch the badge exists to prevent. It now counts
rendered items.
Add Relationship expands the card before opening the modal, so a new row cannot
land behind a closed caret and read as the add having failed.
Verified on screen: machine reads 2 and matches, printer reads 4 and matches.
Frontend 280 passed.
TWO STRIPS OF CHROME SAYING THE SAME KIND OF THING
The machine page opened with a banner - "Dual-bay machine - sibling bay: 2022" -
directly above a hero whose band said "Controlled by". Both answer "what am I
looking at", and splitting them across a banner and a band cost a strip of
vertical space to say so.
The sibling is now a chip in the band beside the controlling PC. The band takes
an `extraitems` prop rather than learning what dualpath is: the pairing is
resolved by the machines plugin, not from the relationships endpoint, so core
cannot derive it, and a dualpath-shaped prop on a core component would be the
machines plugin leaking upward. A page passes [{label, links:[{text, to}]}] and
the band renders it with its own markup, which also keeps the chips visually
identical to the ones it derives itself.
The label stays "Dual-bay sibling" rather than "Dualpath". The banner spelled
the meaning out because a dualpath pair is ONE physical machine in two bays; a
chip reading "Dualpath" would have dropped that.
THE GAGE PAGE FILTERED FOR A COUNTERPART THAT CANNOT EXIST
MeasuringToolDetail was wired counterparttype="machine" labelled "Used on".
Wrong on both counts. The import synthesizes one measuring-tool asset per
metrology PC and a 'controls' edge PC -> tool (stage_metrology), so a gage's
counterpart is a COMPUTER, exactly as on the machine page. Filtering for a
machine peer meant the band rendered nothing on every gage that has a PC.
It looked merely empty rather than wrong because this dev database holds one
measuring tool with no relationships at all, so there was nothing to be missing.
A hand-held gage genuinely has no such edge and correctly shows nothing; a
metrology station now shows the PC that drives it.
The relationships card stays. It is the only surface with Add Relationship and
the per-row delete, and it lists types the band filters out - the band is
triage, the card is management. It does now repeat what the band shows, which
is worth revisiting separately.
Verified on screen. Frontend 280 passed.
Three defects broke the loader's own documented resume guarantee. All three
were silent: nothing raised, the run finished cheerfully, and fewer rows
arrived than anyone counted.
THE PAGE SIZE WAS CLAMPED, SO THE 409-RESOLVE SEARCHED THE FIRST 100 ROWS
_upsert asked for ?per_page=10000 and read one response. The server clamps any
page size to MAX_PAGE_SIZE, which is 100 - not an error, not a warning, just
100 rows. Anything past that resolved to None, so the id never reached the
crosswalk and the run quietly dropped a model's link on every machine that used
it, or an application's installs.
The target already holds more than 100 models and more than 100 applications,
so this fired on any re-run or resume - exactly the case the loader documents
as safe.
_resolve_existing now tries the exact natural-key filter the API has exposed
all along (?modelnumber= on models, ?appname= on applications, ?vendor= on
vendors) and then pages properly, bounded. An endpoint without such a filter
ignores the unknown parameter and the paged walk finds the row.
stage_metrology repeated the same mistake in its own way: it skipped 409s
entirely, so a resumed run left the tool out of the crosswalk and the Controls
edge unmade - metrology PCs driving nothing. stage_printers already had the
resolve branch; that stage was written later and missed it.
THE CROSSWALK WAS NOT BOUND TO THE DATABASE IT DESCRIBED
idmap.json maps this site's legacy ids to assetids in ONE database, and those
assetids mean nothing anywhere else. Keeping it across a drop-and-rebuild left
warranties and printer assignments attaching to whatever asset now holds that
id: wrong data, no error. The runbook's two "drop and rebuild" steps never
mentioned the file.
The loader now writes a uuid marker into the TARGET's own settings table, so it
dies with the data, and stamps it into the crosswalk. A crosswalk built against
a different database is refused before a row is written. The database name
alone would not do - a rebuild keeps the name. A file written before this guard
is adopted rather than refused, so no existing crosswalk is stranded.
Also adds --idmap, so one checkout can drive more than one target.
THE SOURCE DATABASES ARE NAMED, NOT HARDCODED
'shopdb_src' appeared at 29 call sites, so aiming the loader at the prodscratch
restore meant editing run.py. Those names are now LOGICAL and resolved in
Source.db() from SOURCE_DB_MAIN / SOURCE_DB_USB / SOURCE_DB_EMPLOYEES, with
host, port, user and password overridable too. The call sites are untouched -
resolving centrally avoids missing one, and the literals now read as
documentation of which source a stage draws from. The run prints every source
it redirected, so "it imported nothing" can be told apart from "it read the
wrong database".
This is what stood between the loader and the prodscratch run.
TWO SMALLER ONES
_source_password() raised IndexError rather than its intended message whenever
the DATABASE_URL user is not root, which it will not be on a real server.
main() returned 1 on failure into a caller that discarded it. A run that logged
200 endpoint errors exited 0 and read as a success in a deploy log; it now
exits non-zero.
TESTS AND DOCS
The loader had no tests at all. Sixteen now cover the resolve, the crosswalk
binding and the source aliasing - the logic that can run without a MySQL
source, which is the part that was wrong. Verified they are not vacuous by
running the old algorithm against the same fixture: it returns None where the
new one returns the row.
The README claimed 15 stages "complete" with counts from 2026-07-13 that two
later routing changes invalidated, and the runbook told operators to verify a
fresh import against them. Both corrected, with the three gaps NOT fixed here
recorded plainly: warranties, notifications and knowledgebase POST without a
duplicate check so resuming across them doubles rows; there is no --dry-run or
target guard; and the verify stage counts target assets against source machines,
which is apples to oranges.
Full suite 2178 passed.
THE RECIPROCAL LINKS BUG
A PC's page showed "Serves machine 7803" at the top. The machine's page showed
nothing equivalent, though the same edge was right there in the relationships
card at the bottom. That asymmetry read as missing data and was reported as
such.
It was neither missing data nor an API fault. The endpoint returns both
directions and always did; AssetTriageBand already walked outgoing AND incoming
and deduped, with a comment explaining that production holds reciprocal rows.
It then discarded every peer whose asset type was not 'machine', and hardcoded
the label "Serves machine". So the component could only ever work from the PC
side, and it was mounted only on the PC page.
The counterpart type and label are now props, defaulting to the old values so
the PC page is unchanged. The machine page mounts the band with
counterparttype="computer" and the label "Controlled by".
Eliminated with evidence rather than assumed, because this bug had already been
misdiagnosed twice: the endpoint runs two queries and returns both arrays; the
generic card renders both; that route has no pagination to clamp; and a single
permission gate cannot admit one direction and not the other.
THE RELATIONSHIP TYPE WAS ALSO HARDCODED
Not every asset pair is linked by 'controls'. A machine reaches a printer
through usesprinter and defaultprinter, so the band was blank on the printer
page - the one page where the link is not obvious from the asset itself. The
types are now a prop, and the printer page passes its own.
THE CHIPS WERE DEAD LINKS
Found while adopting the band more widely. It linked to /assets/<assetid>.
There is no /assets/:id route and no catch-all, so every chip in the band was
dead - and this change would have propagated that to four more pages.
AssetRelationships.vue already resolved this correctly and privately. That
function is now frontend/src/utils/assetRoute.js and both components use it, so
a third caller cannot get it wrong again. Two things it gets right that are easy
to miss: the detail routes key on pluginid, not assetid, so passing assetid
opens the wrong record; and the lookup is now case-insensitive, because
csvimport.py seeds 'Computer' and 'Machine' capitalised while this site's
database holds them lowercase - a case-sensitive match produced dead links at
whichever site seeded the other way.
The band now carries pluginid and assettypename on each chip, which the
resolver needs.
TESTS
assetRoute had none and neither did the band. Seven cover the resolver,
including the assetid-vs-pluginid confusion and the seeding-case difference.
On the backend, nothing asserted that the TARGET of a directional edge receives
it in `incoming` - only that rows carry isdirectional. That missing assertion is
this bug's exact shape, so it is now pinned: the endpoint was right all along,
and the next such report should not cost anyone a re-diagnosis of the API.
Verified on screen: the machine page shows "Controlled by G6K76CW3ESF" in the
hero, the PC page is unchanged, and a page with no such link renders no empty
band. Full suite 2162 passed, frontend 280 passed.
docs/PLUGINS.md listed sixteen of the seventeen bundled plugins. The missing one
was endpoints - the newest, and the one another site is most likely to ask
about, since it is the Fleet console and the live panels on a PC detail page.
The row records the two things that are not guessable from the name: it depends
on computers because an endpoint is telemetry ABOUT a PC asset rather than a
second record of it, and its four permissions are split deliberately. Read is
grantable to guests by the .view suffix rule; fleet, command and restart are
not, and the plugin declares no override, so the command channel cannot be
opened to anonymous callers by ticking a box.
Ingest is called out as NOT being a route here - agent reports arrive on the
generic collector route under a service token scoped collector.ingest.endpoints.
Somebody looking for the ingest endpoint in this plugin will not find one.
applications/ stays absent on purpose: it has no manifest and no plugin.py, so
per the manifest-less rule it is core frontend surface that always ships, not a
plugin a site can choose.
requirements.txt carried a hand-written header explaining why the lockfile is
universal and hashed, what each flag buys, and a warning not to hand-edit it
because waitress and tzdata were once added by hand and vanished on the next
compile.
Running `uv pip compile` to upgrade two packages wiped all of it, including the
warning about hand-edits vanishing. The header had been documenting its own
failure mode and still lost.
The explanation now lives in requirements.in, which is the file people edit and
which a compile never rewrites. The header is restored in requirements.txt for
anyone reading that file directly, but it is no longer the only copy.
Also records the --upgrade-package form for moving one package without moving
everything else, and the scan_sbom.py check to run before a release.
scan_sbom.py reported 50 advisory matches against shipped components, 16 of them
at or above HIGH. This clears all of them: the scan now reports 0 shipped
findings and the gate passes.
pyjwt 2.12.1 -> 2.13.0
idna 3.13 -> 3.19
axios 1.13.2 -> 1.20.0 (29 advisories)
postcss 8.5.6 -> 8.5.26
dompurify 3.4.11 -> 3.4.14
nanoid 3.3.11 -> 3.3.18
follow-redirects 1.15.11 -> 1.16.0
form-data 4.0.5 -> 4.0.6
Every one is an in-major bump inside a range the manifests already allowed, so
this is a lockfile refresh rather than a migration. The Python side was compiled
with --upgrade-package for each name so nothing else moved; the diff is two
version lines and their hashes.
nanoid is worth recording: it is transitive under postcss at ^3.3.11, and its
advisories are fixed on both the 3.x and 5.x branches. 3.3.18 is the fix that is
actually reachable.
The declared floors for axios and dompurify were raised to the versions in the
lockfile. They were ^1.6.0 and ^3.4.11, which still nominally admitted the
vulnerable range - npm resolves to the highest match so an install was never
going to pick one, but the manifest should state the minimum safe version rather
than rely on that.
axios 1.20 adds optional proxy-agent support, which pulls https-proxy-agent and
agent-base into the tree. Both are Node-only and neither reaches the browser
bundle; the build was checked for them. The main chunk grew 256kB to 271kB
(85.7kB to 90.5kB gzipped) from the rest of the axios delta.
The PyJWT entry in the allowlist was removed rather than kept. It stopped
matching the moment the upgrade landed, and an allowlist that accumulates
entries for advisories that no longer apply is one nobody rereads. The reasoning
is preserved as the worked example in docs/DEPENDENCY-SECURITY.md.
Verified: full suite 2161 passed, frontend 273 passed, build clean,
npm audit --omit=dev reports 0 vulnerabilities.
The rollup took the highest fix version across a component's advisories. That
is wrong whenever an advisory is patched on more than one release branch.
nanoid is the case that exposed it. Both advisories against it are fixed on two
branches - 3.3.16 and 5.1.16, 3.3.18 and 5.1.6 - and the rollup recommended
5.1.16 with a MAJOR warning. But nanoid is transitive, pulled in by postcss at
`^3.3.11`, a range no 5.x release can satisfy. The recommendation was not merely
a bigger job than necessary, it was unreachable: the actual target is 3.3.18, a
patch inside the existing range.
Per advisory the target is now the lowest published fix above the version in
use; per component it is the highest of those, since one upgrade has to satisfy
every advisory against it. Fixes on older branches are skipped rather than
offered as a downgrade - axios advisories list 0.30.x fixes that would have been
a downgrade from 1.13.2.
This also gives the MAJOR label meaning. It now fires only when no fix exists on
the current branch, rather than whenever some branch happened to be higher.
START-HERE routes by the question somebody arrived with, and 'are we exposed to
this CVE' had no entry - the SBOM was only mentioned in the Windows update doc,
which is not where anybody asking that would look.
The SBOM was generated and installed onto every server, and nothing read it.
"Are we exposed to this advisory" was still a person going and looking, which
meant it happened when somebody remembered to.
scripts/scan_sbom.py queries OSV.dev by purl - no account, no API key, no agent,
no vulnerability database to maintain on a box that cannot reach the internet.
The SBOM is already the input OSV wants, so this is a lookup against what
shipped rather than a second scan of the tree that could disagree with it.
Three decisions worth recording:
Shipped and build-only findings are both reported, but only shipped ones can
fail a build. Two thirds of the components here are build tooling the SBOM
already marks scope 'excluded', and the highest-severity match in the whole tree
today is a CRITICAL against the test runner. A gate that fails a release over
that trains everybody to bypass it, and then it catches nothing.
Severity is taken from the advisory, never derived. Where there is only a CVSS
vector the report says UNKNOWN, which is shown and never gates. A guessed score
is worse than an honest gap because somebody would act on it.
The allowlist is a record rather than a mute button: each entry carries the
reason, the accepter, the date, and what would invalidate the reasoning. The
first entry is a worked example - a PyJWT algorithm-confusion advisory that
needs both mixed algorithm families and a raw JSON JWK as the key, and this
application configures neither.
The report ends with a per-component upgrade rollup, since that is the part
somebody acts on. A target on a different major version is labelled rather than
listed beside patch bumps as though the two cost the same.
Current state: 8 shipped components have fixes available, none of them
exploitable through a path this application opens today.
Three colour faults on the PC page, each hiding the next.
ACTIVITY BADGES WERE ALL GREY. The list renderer reads a badge colour from a
ROW FIELD, and the activity panel declared none, so every severity rendered
identically. The feed's whole job is to make a critical event visible and it
was painting it the same as a logon. The route now emits severitycolor - info
grey, warning yellow, critical red - and the panel declares it.
PAUSED AND THE TRANSITIONAL STATES WERE RED. statecolor was green-or-red-or-
grey, so a paused service looked like a stopped one. They are different: a
paused service was paused deliberately, and one stuck in Start Pending is
failing to start on a retry cycle - which is exactly what raises event 7011,
the thing behind the critical counts on the Fleet page. Both are amber now.
NAMED COLOURS ALWAYS GOT WHITE TEXT. readableText only parsed hex and returned
white for anything else, but panels emit NAMES. Green, red and grey are dark
enough that white worked by luck; amber is not, so the fix above shipped as
white-on-yellow and could not be read. Names resolve to hex before the contrast
test, so this holds for any named colour anywhere in the app rather than these
two panels.
colorStyle had no test file at all. It has one now.
Every one of these was found by looking at the rendered page. None of them
would fail a test that checked markup.
Full suite 2131 passed, vitest 273 passed.
Reported from the Activity card: the header dot was red while every visible row
was info, because the critical event was older and sat further down the list.
The dot and the list appeared to contradict each other, and the only way to
find the critical was to scroll.
The dot is computed from all rows the panel received, so a red dot always means
a critical IS in the data. The chips are computed from that same array, which
means the count sits directly under the dot and the two cannot disagree by
construction: the card now reads 'Critical 2 Warning 1 Info 12', and one
click surfaces them wherever they sit in time.
Derived from the panel's EXISTING status declaration rather than a new one per
panel. A card that shows a health dot has already said what it dots on, so
Tracked Services and Tracked Processes get chips with no plugin change, and any
future list panel with a status rollup does too. Two declarations of the same
thing would eventually disagree.
Details that matter: a colour-keyed panel is labelled by meaning, so services
read Critical and OK rather than Red and Green; chips are ordered worst first
rather than alphabetically; and no selection means show everything, not hide
everything - a card that emptied when you deselected the last chip would read
as broken. Filtering to nothing says so, and gives the hidden count.
vitest 267 passed, including the exact reported case.
A site running external mode keeps its devices and check-in/out history in a
separate cmmc_usb database. There was no way to bring that in, so the choice
was to keep the second database forever or lose the history. This adds
'flask usb import-external' - dry run by default, --commit to write - which
reads through the same connection settings the routes already use and never
writes to the source.
Two things it has to reconcile, and they are why it is a command rather than a
few INSERTs.
The schemas disagree about SHAPE. External is an event log, one row per action;
self-hosted is a paired row carrying both times. Pairing walks each device in
time order, and real logs after years of use are untidy. The rule is that
nothing is invented and nothing vanishes: a drive still out becomes an open
checkout and is NOT flagged (that is just a drive somebody has), a return with
no matching issue is skipped and named, and two issues in a row leave the first
open rather than inventing a return time nobody recorded.
Identity is the other. usbdevices.serialnumber is unique and NOT NULL and the
external devices table has no serial, so device_id becomes it. Blank or
duplicated ids stop that device rather than inventing a key. external
device_owner has no column here, so it is appended to the device notes rather
than dropped.
Also adds usbcheckouts.scannedviruses, the one field with nowhere to land.
waswiped already means what external calls sanitized - selfhosted.py was
mapping it - but scanned_viruses was HARDCODED None in the log output: the API
promised the field and had nothing behind it, so an imported history would have
rendered blank. It now reads and writes properly.
Nullable, never defaulted to false. An unrecorded scan is not a failed scan,
and for removable-media records those are very different claims.
Full suite 2131 passed; seven new tests pin the pairing rules.
Reported from production: health showed '418 critical' while the ten most
recent events were all info. Both were true, which is the problem.
The health rollup counts critical events in a 48h window; the activity feed
shows the fifteen most recent events of ANY severity. A box that logged 418
service timeouts overnight and ordinary logons since produces exactly that
pair - a page contradicting itself with no way to tell why.
Two fixes.
The rollup now names the DOMINANT event instead of totalling: 'Service timeout
x418' rather than '418 critical'. Counting by event id costs nothing (one more
GROUP BY column) and turns a number into a lead. This is the same reasoning
that made health a phrase rather than a score; it stopped one step short.
The activity feed now fetches the newest criticals separately and merges them,
and keeps up to five of them when trimming to the cap. They hold their place in
time but survive the cut, so what health is reporting always appears in the
feed that exists to explain it.
Verified against the reported shape: 20 service timeouts two hours old under 12
newer logons. Before, the feed was fifteen logons; now it carries five critical
rows and health reads 'Service timeout x20'.
Worth noting the underlying signal is real - several hundred criticals on one
endpoint in 48h is a box crash-looping or a service failing on a retry cycle.
The display was hiding it, not inventing it.
Full suite 2124 passed, naming and parity gates green.
Select endpoints, then Collect now or Restart across the selection. Restart is
gated on endpoints.restart, both are re-checked server-side.
The confirmation names the MACHINES, not just the hostnames:
Restart 2 computer(s)?
GJX9B2Z3ESF, HOST-7
This STOPS the machines these PCs serve:
DT Office
A dialog listing eight hostnames tells an operator nothing about what they are
about to halt, and that is the whole consequence of the action on a shop floor.
Where no machine is recorded it says so, because 'none listed' and 'we did not
check' are different things at 6am. Only possible now the Machine column exists.
A 409 is reported as a SKIP, not a failure. The queue permits one command in
flight per endpoint, so a bulk action across eight PCs where two are already
busy says '6 queued' and '2 skipped - a command is already in flight'. Folding
those into an error count would make a working button look broken.
Selections clear on every reload: the rows they referred to may not be on the
new page, and acting on an invisible selection is how bulk tools surprise
people.
No per-row buttons. They were built first, and a tenth column on a nowrap table
pushed itself off the edge - selecting one row costs a single click and leaves
one command path instead of two.
Full suite 2124 passed, vitest 260 passed, naming gate green.
The tiles were 6px while .card is 0.25rem, which left Fleet marginally rounder
than every other page. The global card treatment is a separate decision and has
been deferred; until it is made, this page should look like the rest of the app
rather than half-way to a mockup.
Four things the mockup showed and the first cut did not have.
FLEET
- A 'with critical events' tile, from the stability route that existed and
nothing called. Clicking it narrows to rows whose health is not ok - client
side, because the health rollup is assembled per row after pagination, so
it narrows the PAGE rather than the fleet. Worth knowing before the count
is read as fleet-wide.
- PC type and Location filters. The list route already accepted both; there
was simply no control. Both use listAll, not list: perpage is clamped to
100 server-side, so a long location list would lose its tail in a dropdown
while still looking complete.
- The machine cell shows number and name together where they differ, so
'3105 Okuma' rather than a bare number.
PC DETAIL
- A Policy card, contributed by the computers plugin. It matters because the
effective policy already narrows the command allowlist, so without it the
reason a control is unavailable on a given PC is written down nowhere.
Provenance, so it sits in the reference column, not the triage band.
Its route returns an EMPTY field list rather than the feature-gate 404 when
policies are disabled. The panel contract hides a card with no fields, so
it disappears on sites that have not enabled the feature - a 404 would have
rendered a broken card on most of them. That was flagged in the design
review as a snag; this is the fix.
- Refresh Status and Restart Computer moved out of the page header into the
triage band. They act on the MACHINE; Print Label, Edit, Retire and Back to
List act on the RECORD, and mixing them put the most consequential button
on the page - it halts whatever machine the PC serves - between printing a
sticker and editing a form. In the band it sits beside the liveness chip
and the machine it would take down.
The band takes them through an 'actions' slot rather than knowing about PC
verbs, so a measuring tool can put 'Calibrate' there without the core
component learning anything about computers.
Full suite 2124 passed, vitest 260 passed, gates green, spec at 333 paths.
Two changes to the PC page, both about what a responder meets first.
LIVE STATE FIRST. The contributed panels moved to the head of the card flow.
The grid is a two-column FLOW (column-count, with display:contents on the
column wrappers), so a card cannot be pinned to a side and only document order
decides where it lands - putting the panels first fills the left column with
endpoint status, services, processes, interfaces and activity, and leaves
identity, hardware, network and location to the right. Panels still self-order
among themselves by declared position and still self-hide when no plugin
answers for the asset type.
ONE ACTIVITY FEED replaces the Recent events card. Events and commands answer
the same question - what happened to this box - and reading them as two lists
meant interleaving by eye. GET /api/endpoints/asset/<id>/activity merges them
newest-first, capped at 15. Both are keyed by assetid, so it is two indexed
reads and a merge sort rather than a join, and it renders through the existing
ADR-010 list renderer with no bespoke frontend work. A failed command is marked
critical, because that is the row somebody needs to notice. Not collapsed by
default, unlike the card it replaces.
The core AuditLog is deliberately NOT a third source: its rows carry the action
as the entity name and the asset only inside a details string, so including it
would mean parsing prose to find out which machine a row belongs to.
Requester names are withheld from callers who cannot themselves issue commands,
via caller_holds. A read-only viewer - a guest, on a site that grants this
panel - gets the history without a roster of who restarted what, and the lookup
is skipped entirely rather than fetched and discarded. requestedby holds a user
ID, so names resolve through one batched query, not one per row.
Verified on dev telemetry: a queued command shows as 'Collect now / Pending by
admin' above the Windows events, ordered by time.
Full suite 2124 passed, vitest 260 passed, naming/contract/parity gates green,
spec regenerated at 332 paths.
Adds a shared triage band to the PC detail hero: is it alive, what stops
without it, how do I reach it.
'Serves machine' is the reason it exists. On a shop floor a PC matters because
a machine stops without it, and that fact was buried in the generic
relationships card at the bottom of the page. Both edge directions are read,
deduplicated, and filtered to assets typed 'machine' - production holds
reciprocal rows, so reading one direction shows the link on half the pages and
not the other half.
The band is a CORE component, not a PCDetail block. Five asset detail pages
already rebuild the same hero and grid independently, which is how they drifted
apart; machines, printers and network devices can drop this in rather than
growing a sixth copy. Liveness is passed in rather than fetched, because only a
plugin that collects telemetry can answer it and a site may not have one.
The status chip's verdict comes from the SERVER: /api/endpoints/<id> now
returns the same reporting/quiet/never it computes for the fleet list. The
quiet window is a site setting, so a threshold guessed in the browser would
disagree with the Fleet list about the same endpoint the moment a site changed
it.
Two cards removed from the body:
- Remote Access duplicated the band's chips. It is what a responder clicks
first, so it belongs above the fold, once.
- Status was worse than duplication. It reads the COMPUTER RECORD, while the
endpoint panel beside it reads live agent telemetry, and on an
agent-managed PC the record fields sit empty - so the page rendered 'Last
Reported: Never' a few inches from live telemetry saying otherwise. It now
renders only when that record actually holds something, which is the case
on sites running the asset-report script instead of the agent.
Verified against dev telemetry: band shows 'Reporting, 1 h ago', the machine it
serves, and its access chips; both duplicate cards are gone.
vitest 260 passed, naming/contract/parity gates green.
Two colour bugs on the same page, both invisible in light mode.
A button does not inherit color the way it inherits font, so the plain
Endpoints total rendered in the browser default black on the dark theme's card.
The coloured tiles set their own color, which is why only that one showed it.
The coloured ones were wrong too, just less obviously: dark mode redefines
--success and --warning to bright variants but leaves --success-dark and
--warning-dark tuned for a white background, so those numbers and the health
pills came out dark green and dark amber on a dark blue card.
Scoped to the stamped [data-theme="dark"] rather than a prefers-color-scheme
query, matching the note in style.css: the theme store always stamps the
attribute, and a bare media query leaks dark styling into light mode on a
dark-OS machine.
Twelve endpoints routes shared one permission, which made a reasonable policy
impossible to express. A site that wants signed-out visitors to see live status
on a PC page had to hand them the entire fleet - including every logged-in
username on the floor - because the per-asset panels and the fleet-wide list
were gated on the same string.
endpoints.fleet now gates the list and the three dashboard routes.
endpoints.view keeps the per-asset reads: status, services, processes,
interfaces, events, command history.
The name is load-bearing. guest_grantable_permissions() opens any '.view' or
'.read' permission to the guest role, so calling this one 'endpoints.fleet'
rather than 'endpoints.fleetview' is what keeps the roster off the public
surface while per-asset telemetry stays grantable. Verified both ways.
Also: the services panel emitted Start/Stop row actions to every reader,
because row_actions_for() never consulted the caller. Clicking was always
refused server-side, so nothing was exploitable, but showing a read-only viewer
controls over a shop-floor PC is the wrong signal - and it stops being cosmetic
the moment a site grants that panel to guests.
That needed a contract addition rather than a local check: caller_holds() on
shopdb.api answers whether THIS caller holds a permission without refusing the
request, which is the complement to require_permission for a surface that
serves everyone but should only offer what the caller can act on. Guests are
included, since a signed-out visitor holds whatever the guest role was granted.
Reaching into shopdb.utils.authz instead would repeat the ADR-002 violation the
contract gate caught earlier in this branch. Contract 0.24.0 -> 0.25.0.
DEPLOYMENT: endpoints.fleet is a new permission. Run 'flask seed permissions'
and grant it to the roles that should keep seeing the Fleet page. Until then
that page is admin-only, since admins bypass every check - a safe direction,
but not an automatic one.
Full suite 2124 passed, naming and contract gates green, map and spec
regenerated.
The fleet list showed hostname, user, uptime, CPU, memory and last-reported,
sorted by last-reported descending. Two of those columns have never held a
value, and that sort floats the healthiest endpoints to the top and buries the
ones somebody needs to act on.
What a shop-floor fleet view is actually for is which MACHINE stops if a PC is
dead, so that is now a column. Both edge directions are checked, deduplicated,
and constrained to assets typed 'machine' - without that last filter the column
filled up with office locations that happen to carry a controls edge.
Also here:
- Sorted WORST FIRST: never reported, then quiet, then reporting. Expressed
as a sortable SQL rank rather than a label applied after the fact, so the
order survives pagination.
- Health rollup as a PHRASE, not a score: 'eDNC stopped', '2 critical'. A
percentage does not tell an operator what to do.
- Filters for status, PC type, location and business unit; a sort parameter
(the route accepted none); and search widened from hostname alone to asset
number and the machine number a PC serves, which is how people actually
look a bay up.
- CPU and Memory columns dropped. The agent declares those fields and has
never assigned them, so they rendered '-' on every row.
- A summary strip wired to the dashboard routes, which existed and which
nothing called.
Two bugs found while building it:
The health rollup first read a 'running' boolean, but service rows carry a
STATE STRING and the vocabulary varies by source, so every service would have
reported as stopped. The panel and the rollup now share one _service_running()
helper; two copies of that rule would drift.
The summary counted quiet as total - reporting, which folded never-reported
endpoints in with quiet ones. Those are different failures - an enrolment or
key problem versus a machine that went down - so 'never' is counted separately.
Per-page batching for machines and critical-event counts: three queries per
page rather than two per row.
Full suite 2124 passed, vitest 260 passed, naming and contract gates green.
get_pagination_params reads 'perpage' and only falls back to 'per_page', and
this client has always sent 'perpage', so the interceptor's duplicate never did
anything except make every list URL carry a parameter twice.
Removing it also clears a live production problem. The endpoints list route used
to answer a 308, which is a PERMANENT redirect: the browser caches it against
the exact URL including its query string. Clients that hit the old route keep
that rewrite after the server is fixed, because the request no longer leaves the
machine - nothing deployed can reach it, and it fails as mixed content when the
cached target is http. Confirmed in the field: the page failed on load with the
default query string while a search, whose query string differs, worked in the
same session.
Changing the query string changes the cache key, so an affected client misses
the poisoned entry and reaches the corrected route without the user having to
clear site data.
Plugin views that pass per_page explicitly still work - the backend fallback is
untouched. vitest 260 passed.
An endpoint reported telemetry correctly for hours while the Fleet page showed
"No endpoints found". The PC detail page displayed its panels the whole time, so
ingest was plainly working; only the list was empty.
The list route was the one route in 331 paths registered with a trailing slash:
@endpoints_bp.route('/', methods=['GET']) -> /api/endpoints/
The browser client calls it without one, so Flask answered a 308 redirect to an
ABSOLUTE url. On dev that resolves and nobody notices. Behind the IIS subpath it
cannot: with no ProxyFix, Werkzeug builds that Location from the scheme and host
it believes it has, so it emitted http:// and dropped the /shopdb prefix. A
browser will not follow an https page to http, the request failed, and the
page's catch block rendered the empty state.
Every other route in the plugin worked throughout, because none of them carries
a trailing slash - the detail panels call /endpoints/asset/<id>/status and were
never redirected. Detail works, list empty is the signature of this bug.
Registered as '' now, so the rule is /api/endpoints. That is what
CONTRIBUTING.md's API section spells and what every other list route in the
product already did. The inventory and the generated spec are updated to match.
Also here:
- tests/test_core/test_no_redirecting_routes.py fails the build if any /api
rule is registered with a trailing slash and strict slashes. Verified by
planting the violation: it named the offending route, then passed once
reverted. It takes the shared app fixture rather than building its own
application - doing that rebinds SQLAlchemy and leaves the NEXT module
without tables, which showed up as three unrelated "no such table: settings"
failures visible only in a full run.
- The Fleet page now renders "Could not load endpoints (HTTP nnn). This is a
failed request, not an empty fleet." A list that reports a 403 and an empty
fleet identically is what made this take as long as it did.
- TRUSTED_PROXY_COUNT (default 0) applies ProxyFix when set. Off by default on
purpose: it makes X-Forwarded-* trusted, and a client can send those itself,
while the login rate limit and account lockout are keyed on the client
address. Set it to 1 on an instance behind IIS.
Full suite 2124 passed, vitest 260 passed, naming and docs gates green.
The export died with no output and no dists, exit 1, having committed nothing.
grep exits 1 when it matches NOTHING, and under set -euo pipefail that failing
pipeline takes the script with it. Once every internal reference had already
been scrubbed, the gitea grep matched nothing, and the SUCCESS case became the
abort. It printed not one line explaining itself, which is what made it cost an
afternoon: the run looked like it had worked.
All three scrub greps are guarded now. This is the third failure of this shape
in this file, after the two fetch/push tag omissions.
Also excludes a stray .env copy from the sync. A hand-made .env.bak-<date>
reached the publication tree, because rsync walks the WORKING TREE and
.gitignore does not apply. The scrub gate caught it, which is what that gate is
for, and it never reached a commit or a bundle - but a file of live credentials
should not get that far. --include '.env.example' comes first so the example
still publishes.
Both verified by planting a violation: a no-match grep under pipefail, and a
planted .env.bak file that the sync now leaves behind.
The endpoints plugin declared a section of its own, so the sidebar rendered a
one-item 'Fleet' group at the bottom, below Information, far from the PCs the
view is about. Its own docstring said it belonged just after PCs, and its
position (16, against PCs at 15) already said so too; only the section
disagreed.
Filed under assets, which puts it directly under PCs. The now-empty fleet
section is removed from NAV_SECTIONS - an unused section is an invitation to
file something else there.
The printers observedqueues migration failed with error 1071 on a 5.6 dev
server whose file format was still Antelope. The unique index it needs is
(hostname, queuename), two utf8mb4 VARCHAR(255) columns, which is 2040 bytes
against a 767-byte cap.
The compiler hook in shopdb/utils/mysql_charset.py was already emitting
ROW_FORMAT=DYNAMIC, which raises that cap to 3072. What is easy to miss is that
Antelope makes 5.6 IGNORE the row format and build a Compact table with a
warning rather than an error, so the tables look created and keep the old limit.
docs/DEPLOY.md has required innodb_large_prefix plus Barracuda for pre-5.7
servers all along; nothing pointed a developer hitting 1071 at it.
So the troubleshooting row that said 'use MySQL 8 for dev' now gives the fix,
names the silent Compact downgrade, and says such tables stay capped until they
are rebuilt with ALTER TABLE ... ROW_FORMAT=DYNAMIC.
DATABASE_URL also gains ?charset=utf8mb4, which DEPLOY.md already requires and
this environment did not have.
Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.
What arrives:
endpoints plugin - a ShopDB agent reporting endpoint state, telemetry,
services and interfaces, with a command channel the agent
long-polls. Ingest stays on the generic collector route
(ADR-006); the agent key is a UI-managed single-scope
token whose revocation is an instant kill switch.
computers policies - Configuration and Business Policies (RMM-style), scope
trees validated at write time, actions checked against
the endpoints command allowlist. Feature-gated: every
route 404s while the setting is off, so enabling it can
never half-expose the surface.
asset CSV import - templates, a collector script carrying the site path, and
a validate-then-commit import. A plugin can own a virtual
column plus a post-write row hook, which is how a PC's
machine number becomes a controls link without a
hand-built relationships sheet.
agent/ - the Windows agent and its installer.
Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.
MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).
Review fixes carried on top:
- six policy GETs were jwt_required(optional=True), which authenticates a
caller without checking anything; they require computers.view now, and six
redundant jwt_required decorators are gone
- the agent installer took a default BaseUrl and CollectorKey, so a mistyped
run silently pointed an endpoint at the wrong instance; both are mandatory
and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
- a committing CSV import wrote no audit row
Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.
Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.
Every job on GitHub is warning:
Node.js 20 is deprecated. The following actions target Node.js 20 but are
being forced to run on Node.js 24: actions/checkout@v4, actions/setup-node@v4
Working today because the runners shim it, and a hard failure the day that shim
goes. checkout@v5, setup-node@v5 and setup-python@v6 are the releases built for
Node 24, so the warning goes away rather than being suppressed.
Nine references across five jobs. The comment above them says why, so nobody
pins them back to the v4 generation on the assumption it is the safer number.
The org's actions policy allows GitHub-owned actions with no SHA pinning
required, so the tag change needs no allow-list update.
Two faults with one cause: docs/ is excluded from publication because the prose
becomes the wiki, and two things under docs/ are not prose.
shopdb/core/api/docs.py reads docs/openapi.json AT RUNTIME to serve /api/docs.
Excluding it meant every site installed from the published repo shipped the
ReDoc page, loaded it, and got {"error":"openapi.json not generated"}. The spec
now publishes - it is a product file that happens to live under docs/. The prose
still goes to the wiki and nowhere else, and docs/api-inventory.json stays out,
because nothing reads it at runtime.
The second fault is the reason this was worth chasing. Three gate modules skip
themselves when the file they read is absent, which is correct in a published
checkout and silently wrong on GitHub, which runs against exactly that checkout.
The modules skipped, CI reported green, and an outside branch added nine
undocumented routes and passed. A gate that stops running is worse than no gate,
because nobody looks at it again.
scripts/check-gates-ran.sh asserts each gate ran, with two lists: the ones that
must run everywhere, and the ones needing files publication deliberately leaves
behind - required here, allowed to be absent there. A `gates` job runs it in CI.
Proven in both directions: green in each checkout, and red when the file a gate
reads is removed.
Also purge instance/plugins.json from the publication tree. It was never
published - it is untracked there - but an old copy had sat in that tree for
months, protected from --delete by its own exclude, and it made the published
repo fail its own suite locally: 13 plugins enabled where the code expects 16.
A publication tree that cannot pass its own tests cannot be verified before
shipping. That is the exact trap the comment at the top of the script warns
about, which had claimed a second victim.
Also: concurrency on the workflow, so a second push supersedes the first rather
than queueing behind it holding a self-hosted runner.
A link on its own invites being pasted onto text that is already in the chat
box, and every chat client turns the selection into the link's caption. The
secret then travels as the visible message with the secure link wrapped around
it, protecting nothing.
This is not hypothetical. Six LAPS passwords went out that way in production -
machine names and passwords as the caption, the correct /s/<token>#<key> URL
underneath, doing nothing at all.
Copy message puts a whole sentence on the clipboard: what the link is, how many
opens it has, when it expires, and then the URL. A sentence has nowhere to
attach itself - pasted into an empty box it is a message, pasted over a
selection it replaces it rather than decorating it. Plain text on purpose; a
rich-text flavour is exactly what would let a secret ride along as markup.
The success card now says to delete the password from the chat box before
pasting. The existing hint explains that the key cannot be recovered, which is
true and does not stop anybody pasting the password itself.
Nothing about the link was wrong and nothing about it changed: the URL carries
the token, the key rides in the fragment the browser never sends, the plaintext
is dropped the moment it is sealed, and redeeming stays a POST so a preview
cannot spend it.
Sending somebody a label meant sending a screenshot and a list of settings to
retype, and they would get one of them wrong. Copy share link hands them a URL
that reopens the generator with the content, the label, the copy count and
every layout value that was actually changed.
The whole payload rides in the fragment. A fragment is never transmitted, so it
does not reach Flask, it is not in the IIS log, and the proxy between them never
sees it - which matters because the content is usually an asset number or a
serial, and those have no business scattered through logs. It is NOT secrecy:
the link sits in a chat message and in browser history, and anyone holding it
can read the content. Secure Send exists for the case where that matters, and
the note under the box says so.
Only values that differ from a first-time visitor's defaults are carried, so
the link stays short enough to paste into a chat box; a shared link then wins
over the recipient's own saved settings, because somebody following one is
asking to see THAT code rather than their own last one. An unknown key is
ignored rather than refused, so a link made by a newer version still opens on
an older one. The picture is never carried - a data URL runs to megabytes.
Values are coerced back to the type the field holds. Everything arrives from a
URL as a string, and a nudge of 0 plus '2' is '02'.
Proven end to end: a link made in one browser, opened in a clean context with
no saved settings, rendering the same code.
It struck the heading and left everything underneath looking untouched, which
is the opposite of what somebody ticking "this section is done" means. The
strike now covers every block under that heading, down to the next heading of
the same or higher rank - so a top-level section takes its sub-sections with
it.
That needed the rank, and the HTML importer was throwing it away: h2, h3 and h4
all arrived as a flat 'heading', so a parent heading had no children to cross
out. It was the visible half of a real loss - a converted article kept its
words and lost its shape. Headings now carry their level and render at it, and
a guide written before this reads as flat, exactly as it did.
Code blocks, tables and images inside a finished section fade but are not
struck. A line through a command makes it unreadable, and re-reading the
command is usually the reason to look back at a step already done.
The five imported guides were re-imported so they carry their structure.
Every tool on that page is usable without an account, on purpose - the routes
carry no requiresAuth and say so - and Secure Send is the one write this
product offers a guest. The sidebar entry still never appeared for them: it
named no permission, so the filter that decides what a signed-out visitor may
see had nothing to go on and dropped it.
The plugin had no read permission to name, only tools.secrets, so it gains
tools.view: reaching the page at all. The tools themselves run in the browser
and touch no data, which is why this is about whether the entry appears rather
than about what anybody can read.
The item names either right. Somebody granted only Secure Send needs the entry
that leads to it, and a manager who holds neither still does not see it.
The sidebar hides what the caller cannot reach - that is what makes it useful
to a guest, and what stops a member with a sparse role being offered links that
can only answer 403. It reads the permission off the nav item to decide.
Only four items ever declared one. Machines, PCs, Printers, Network, USB,
Knowledge Base, Notifications, Calendar, Dashboard, Map, Applications and
Reports declared nothing, so the filter had nothing to go on: granting the
guest role machines.view opened the API and left the sidebar empty, which reads
as the grant not working.
Every item now names its read permission - in the plugins, in the core
navigation endpoint, and in the frontend's fallback list, which is what renders
when the endpoint is unreachable and would otherwise disagree with it.
A sweep of the URL map found 165 GET routes that answered anybody, with no
token at all: the whole asset inventory, every machine, PC, printer and network
device, the USB checkout history and who is holding which drive, the staff
directory with its photographs, every core report. Guest access had been argued
about one feature-flag at a time while the front door stood open.
Each of those now carries the read permission it belongs to. That also makes
the guest role mean something: a site can open exactly the reads it chooses,
and nothing follows by accident.
Thirty-three endpoints stay public, and each is named with its reason rather
than left to be inferred: the app shell and its bootstrap, the settings
allowlist (the handler already filters an anonymous caller down to branding and
a few keys), images fetched by <img src> which carries no Authorization header,
what a display kiosk renders before anybody signs in, the parts and USB kiosks,
the client scripts that run as SYSTEM before a user session exists, and a
secure-send link whose unguessable token IS the authorisation.
tests/test_core/test_public_read_surface.py is the standing gate, the companion
to the mutating-route sweep that has guarded writes for months. A new public
read fails CI until somebody adds it to that list in front of a reviewer. Four
further reads need a login but no permission, because the handler decides from
who you are - your own user record, your own API tokens, the role catalogue the
grid renders - and the same test proves each still refuses an anonymous caller.
Verified not vacuous by ungating one route and watching the sweep name it.
The employees plugin gained employees.view; the directory is names, SSOs,
photographs and reporting lines, and it had no permission of any kind.
Around ninety tests asserted payload shapes against endpoints that answered
anybody. They are about what the answer says, not about who may ask, so they
are signed in now rather than the gate being loosened. The ones that assert an
anonymous caller is refused were left exactly as they were.
Only checklist items could be ticked, which answers "what have I done" but not
the question a long procedure actually raises: where was I. Somebody looks away
to do the thing the guide asked for and looks back at a wall of text.
Every step and every heading now carries a checkbox while reading. Ticking one
strikes it through and fades it rather than hiding it, because somebody
scanning back up the page has to still be able to read what they did. The
number is struck through with it, but the step's own "Step 3" label is not -
that is the address of the thing, not part of what was done.
Headings count, not just steps. A guide converted from an HTML article often
has fifty headings and no steps at all, and those are exactly the guides long
enough to lose your place in. That also means the card's "N steps to tick"
stops reading as a broken feature on a guide that is entirely tickable.
Reading only, as the checklists already were: in the editor a tick would be an
edit, and an author clicking around their own draft would quietly change what
the next reader sees as done. Progress stays per browser and per version -
shopfloor PCs share a login, so one technician's ticks must not greet the next
person on that bay, and step 4 of version 3 is not step 4 of version 2.
Whether somebody with no account may see something was being answered one
feature at a time: a securesend_allow_guests setting, an enable flag per kiosk,
and very nearly a third for guides - each with its own settings page, its own
vocabulary and its own hand-written branch inside a route. Every one of them
answered a permissions question, and this app already administers permissions
per role.
So a caller with no token is not nobody: they hold the seeded `guest` role.
require_permission consults it when there is no identity, which is the entire
mechanism. The role is seeded holding NOTHING, so an upgrade opens no door; a
site opens one by ticking a box in the same grid it uses for every other role,
and closes it the same way. Revocation is read on every request rather than
cached, because taking guest access away is what somebody does the moment they
realise what they opened, and it has to bite immediately.
The redundant @jwt_required() above require_permission is gone from 184 routes.
It was belt-and-braces before; now it is actively wrong, because it refuses a
request with no token before the permission check can have an opinion, so a
grant would appear to do nothing.
What a guest may hold is bounded. Any .view or .read permission is grantable.
A WRITE has to be blessed by whoever owns it, through the new
get_guest_grantable_permissions hook (contract 0.21.0), and Secure Send is the
only case that exists: sending a password to a vendor is the one write this
product means to offer somebody who never signs in. Everything else is refused
by the API and greyed out in the grid rather than warned about - a checkbox
that quietly makes deleting an asset anonymous is not a warning anybody reads
twice. A guest write records no creator: the trail says when and from where,
never who, and that is the reason for the bar.
GE-Enforce and Slides gained read permissions so they can be opened to guests
without handing out the manage right. Their read routes accept either through
require_any_permission, so nothing an existing manager could reach has moved.
The navigation endpoint now returns only what the caller can actually reach.
Without that a signed-out visitor gets a sidebar of the whole app where every
link bounces off the login page, which reads as a broken site rather than as a
closed door. A nav item may name several permissions; holding any one is
enough, which is what GE-Enforce and Slides need now that a viewer and a
manager hold different strings.
Two things this found on the way: current_user is a LocalProxy, so
`current_user is None` is False for a request with nobody behind it - harmless
under a required JWT, an AttributeError the moment verification became
optional. And an image pasted into a guide before its first save was never
attached to it, so it was visible to editors and invisible to every reader;
saving now attaches it, and `flask guides fix-images` repairs what is stored.
The guides guest setting written earlier today is gone. Granting guides.view to
the guest role replaces it, which is the point.
Guides are written on one site and needed on all of them, and the sites that
matter are air-gapped from wherever the writing happens. Until now a procedure
was stranded in the database it was typed into: the HTML importer could rebuild
one from its original article, but anything authored in the editor had nowhere
to go.
flask guides export writes one .guide.json per guide, with every image the
blocks reference carried inside it as base64. One file on purpose - a bundle
that needs its images copied alongside it is a bundle that will eventually
arrive without them. flask guides import reads them back, a directory at a
time, and refuses anything that is not a bundle before it writes a row rather
than half-importing a stray .json sitting in the same folder.
What travels is the content: title, summary, tags, blocks, images. What does
not is ids, revision history, authorship and publication state, because those
belong to whichever database issued them - an imported guide starts as a draft
at revision 1, owned by whoever ran the import. The SLUG does travel: it is the
address somebody pastes into a work order, and the same guide should answer to
the same name on every site.
--replace updates the guide already carrying that slug instead of creating a
second one, which is how a corrected guide is re-shipped. It drops the stored
images first: the incoming bundle is the whole truth for that guide, and a
guide re-shipped monthly would otherwise accumulate orphans nothing points at.
Bundles refuse a version they are too old to read rather than importing the
parts they recognise. A newer site's guide is not a guide this one can be
trusted to render.
Proven by round-tripping a five-guide export back into the same site, including
one carrying two PNG screenshots, and reading it in a browser.
A site adopting this app arrives holding procedures already written: styled
HTML pages that get mailed around, opened off a share, and edited by whoever
still has the file. Those pages are exactly what a guide is for, and retyping
one into the editor is how a good procedure stays on a share forever.
flask guides import-html reads the page and produces blocks. It recognises the
vocabulary those pages actually use - a step box with its own number, an alert
in one of three colours, a code block with a copy button, a checkbox list, a
table with a header row - and maps each to the block that already renders it,
lifting embedded images into guideimages so a document stops carrying its
screenshots as base64. Page furniture goes: the header, the footer, and the
table of contents a guide builds for itself from its own headings.
The conversion is lossy on purpose. Colour, fonts, width and nested layout are
dropped, because storing blocks rather than markup is what makes dark mode,
printing and tickable steps work at all. So an import is always a DRAFT and the
default is a dry run that reports what it would produce and writes nothing.
Blocks go through the same validator the API uses: an import must not be able
to store something the editor could not have produced.
The parser is tolerant, because real articles are not well-formed. An unclosed
<p> is normal HTML and the browser closes it at the next block element; without
the same rule one unclosed tag swallows the rest of the article into a single
paragraph, which is what it did before the test caught it.
Proven end to end against four real articles, including one carrying two PNG
screenshots, read back in a browser.
POST /api/reports/email has always taken any {columns, rows} from any caller,
but the Email report button existed on the toner and warranty pages only - not
because those reports are special, but because they happen to be their own
routed pages and the reports page had no button at all. It is now beside Export
CSV for every report the page can build a table from, which is all of them:
plugin reports through their declared columns, core reports through the flat
list they return, and the asset inventory summary by flattening its three
tables into one with the grouping named in a column. Presentation-only fields -
a status colour, a link id - are left out; a column of hex codes is worse than
no column. The shapes live in a module with tests rather than in the view.
The button is now hidden without reports.export on all three pages. Offering
somebody a button that can only answer 403 is worse than not offering it.
Separately, on the USB kiosk a long drive name pushed Take or Return out
through the side of its card, leaving a drive nobody could press. The name is a
flex item and so defaulted to min-width:auto, keeping its full intrinsic width;
it now yields, wraps mid-word for the serial-style names that carry no spaces,
and clamps its description to two lines so one long name cannot make every card
in the row tall. The locker line is never clipped - it is how somebody finds
the drive.
A cabinet of thirty drives was one long column to read through. The free list
now filters from a search box and lays out in as many columns as the panel is
wide - four on a wall display, two on a tablet, one on a phone - while the
steps that hold a single control stay narrow, because stretching one input
across a 24 inch screen reads as a broken page.
A drive is found by its id, by what is written on it, or by where it lives. The
words can arrive in any order, and "locker" is part of what is searched, so
"locker engineer" finds the engineering locker - that is how people say it out
loud. Filtering is client-side: every free drive is already on the screen, and a
round trip per keystroke on a wall panel would be worse than no filter at all.
The matcher lives in its own module with tests rather than inside the view.
The box takes focus as the picker opens so somebody can simply start typing,
and it is cleared when the kiosk resets - the next person at the cabinet must
not inherit a filter and conclude there are no drives left.
Also: both this view and the guides blocks coloured muted text with a
`--textmuted` variable this app has never defined. The browser dropped the
declaration, so every hint, subtitle and locker line was rendering at full
strength. The variable is `--text-light`.
The reports page ran a hardcoded switch over five core report ids and fell
through to `default: console.error('Unknown report')`, and its results panel was
a matching chain of per-id templates ending in a `JSON.stringify`. So a plugin
could register a report, watch it appear in the catalogue, and get a wall of
JSON when it was opened - which is where the three printedparts reports have
been since they were added.
Any report that declares an endpoint is now fetched from it, and any report
returning {columns, rows} renders as a table. That shape is already the
convention on the server side, so a plugin needs no core code at all to have a
working report. The per-id templates stay for the core reports whose payloads
are not tabular.
Export CSV goes through the api client rather than window.open. A new window
carries no Authorization header, so every report behind a real jwt_required -
which is every plugin one - answered the export button with a 401. The core
reports are jwt-optional, which is the only reason the old way ever appeared to
work. It also followed /api/reports/<id>, which does not exist for a plugin
report, instead of the endpoint the report itself came from.
A report that fails now says so instead of leaving an empty panel, and the page
offers the time window the endpoints already accepted - printedparts-consumption
was permanently stuck on its 30-day default because nothing could send one.
Verified in a browser against both plugin report families and the core reports.
CMMC removable-media tracking is an audit obligation, and answering "who had
this drive, and when" meant paging through the check-in/out list by hand.
GET /api/usb/reports/audit returns the events - a take and a return are separate
rows, because a count cannot answer the question an audit actually asks - with
summary figures riding along for the people who only want the shape of it, and
a CSV for the people who want it in a spreadsheet.
The window is named rather than a day count: today, last month, six months, a
year, all time. Nobody should have to work out that a year is 365.
Both directory modes answer in one voice. The external path reads the
cmmc_usb checkinoutlog; self-hosted expands one usbcheckouts row into its two
ends, so a returned drive is distinguishable from one still out, and the window
filter runs in SQL against either end of the loan - a drive taken before the
window and returned inside it is a check-in the window has to show. Rows come
back from the mode-specific half; the CSV, the summary and the response shape
are built once in the route.
Ordering is total in both modes. Timestamp alone ties for two events in the same
second, and the same query then returns them in a different order each run,
which reads as the log being unstable.
The kiosk was a single column of full-width rows top-aligned on the page, so a
cabinet of a dozen drives meant scrolling a wall panel, and the page itself
scrolled - which takes the "Your drives" heading off screen and leaves somebody
looking at a list with no idea whose drives they are. The lists now sit in a
grid that fills the width and scroll INSIDE themselves against a bounded
height, with the panel centred as one block the way the parts kiosk beside it
already is. Two kiosks on the same floor that sit differently on their screens
read as two different products.
Drive ids sorted as text, which puts USB-10 ahead of USB-3. Compared with a
numeric collator so the list runs in the order the labels on the drives read.
fetchUser() existed and was called from nowhere. The signed-in user - including
the permission list every v-if in the app reads - was written once at login and
cached in localStorage, and nothing ever refreshed it. So an admin granted
somebody guides.create, the person was told they had it, and the New guide
button stayed hidden until they happened to log out and back in. Nothing on
screen explained why, and the natural conclusion is that the permission system
is broken.
Called on app boot, so a permission change lands on the next page load.
Reproduced before and after: button hidden, permission granted mid-session,
plain reload, button there.
Only a 401 logs out now. The old handler logged out on ANY failure, so an API
restart or a network blip would have thrown people out of sessions they still
held - a cached user is stale, not invalid.
The kiosk page tells an administrator to enable it in Settings under USB. That
place did not exist - the setting was reachable only through the API or a shell,
which is the same trap the backups plugin shipped with. A settings page and a
catalogue card, so the instruction leads somewhere.
Machines, PCs, printers and measuring tools all carry Print Label / Edit /
Retire / Back to List in a page header. A network device had no header at all:
Edit was tucked inside the hero title row, and Print Label and Delete sat in a
bar at the very bottom of the page, past every card. Managing one meant looking
somewhere different from every other asset type.
They move to the standard header. The bottom bar is removed rather than left as
a second copy, and the two delete handlers collapse into one - the route is a
soft delete, so it is called Retire here as it is everywhere else, and the
confirm says what is kept.
Four things people kept tripping over.
The move controls were a literal ^ and v character. They are arrows now.
Sections can be dragged. The handle is a grip in the block bar, not the block
itself: making the whole container draggable swallows text selection inside its
own textarea, so you could no longer select a word to bold it.
The type palette lived once, at the bottom of the entire guide. Adding a step in
the middle meant adding it at the end and walking it up one press at a time.
There is now an insert point in every gap - and at the end, so there is one way
to add a section rather than two.
Linking to another guide or a KB article meant knowing its slug or id by heart.
A picker searches both by title and writes the markdown link at the cursor. It
offers published guides only - never send a reader to a draft - and it stores an
in-app path rather than an absolute URL, so the link survives the site being
reached by another hostname or mounted under a subpath. The knowledge base is a
separate plugin that a site may not have installed; its absence costs nothing
but its results.
The section headings were derived from position ranges: 10-29 became Assets,
30-49 became Information. A number is for ORDER; it was never a statement about
meaning, and using it as one meant a plugin landed in whichever bucket its
number happened to fall in.
That is how USB Devices - an asset - came to sit under Information beside
GE-Enforce and Tech Tools, and 3D Parts with it. Nobody chose that. It also
meant two plugins picking the same number silently reordered each other:
Warranties and Guides both held 8, the tie broke on plugin load order, and
Guides rendered under Warranties rather than under Slides where it was asked to
go. Warranties moves to 9.
Each nav item now names its section and the renderer groups by what was
declared. An item with no section stays in the unlabelled group at the top,
which is the safe default: it appears, it is simply not filed.
Assets gains USB Devices and 3D Parts. GE-Enforce and Tech Tools move to a new
Fleet section - they are neither assets nor reference material, and calling them
Information was only ever an artefact of their position numbers.
Also: sorting used `position || 99`, and 0 is falsy in JavaScript, so Dashboard -
the only item that uses 0 - sorted to the end of its own group.
The kiosk route lives outside AppLayout, so nothing linked to it - a full-screen
page with no way in from the app is a page nobody finds. It joins the Displays
section beside Parts Kiosk, TV Slideshow and the Shopfloor Dashboard, gated on
the route existing so a build without the USB plugin does not advertise it.
The people who actually take USB drives are standing at the cabinet, and most
of them have no account here. Check-out and check-in were gated behind a login
that was not doing much anyway: both modes already attribute the action to a
scanned BADGE rather than to the signed-in user, so the login was gating the
screen, not the record.
/usb-kiosk is a full-screen touch page modelled on the parts kiosk, off unless
a site sets usb_kiosk_enabled - every route 404s until then, rather than 403,
so a disabled kiosk does not advertise that there is a door here. It works
against the self-hosted tables or an existing cmmc_usb database.
THE LIST IS THE TRADE. These drives carry no barcode, so unlike the parts kiosk
there is nothing to present and the screen has to offer a list - which exposes
more than a scan-first design, where you had to already hold a drive to learn
anything. So the list is not anonymous: an SSO comes first and must resolve to
a real person before any drive is named, which is the same bar taking a drive
already had to clear. It is a POST so the badge stays out of proxy access logs.
The main screen shows the caller's own drives, what is free, and a COUNT of
everything else; holders of other people's drives appear only behind a second
deliberate tap, so nobody reads the whole cabinet off a wall panel in passing.
A badge that resolves to no employee is REFUSED, unlike the signed-in route
which creates the user row. A checkout nobody can trace to a person defeats the
tracking, and at a kiosk there is no operator standing there to vouch for it.
Returning somebody else's drive asks first and then records the badge that was
SCANNED, so the log says who actually handed it back. That is the honest record
and it needs no change to a schema this app does not own. Refusing a mismatch
outright is a setting but not the default: a drive left behind by somebody on
leave would otherwise be unreturnable at the kiosk, which just moves the problem
to a person with a login.
The authz sweep caught these writes, twice - once for the two mutations and
again for the two list endpoints. Both are exempted with the reasoning written
out beside the parts-kiosk entry rather than silenced. The list endpoints are
POSTs that change nothing.
Verified by driving the real page: SSO on the keypad, tap Take, database moves.
.env is committed git-crypt encrypted. Without the key the checked out
file is the raw encrypted blob, and python-dotenv fails reading it as
UTF-8 with "invalid start byte". The migrations job already provides
DATABASE_URL, SECRET_KEY and JWT_SECRET_KEY through env:, so removing
the file is enough. Previously masked by the job dying earlier on a
missing mysql client.
A service container is reachable on 127.0.0.1 only when the job runs
directly on the runner host, which is how GitHub-hosted runners work and
how the old host-executor runner worked. Under a container-mode runner
the job is itself a container on the services network, published ports
land on the runner host rather than the job's own localhost, and the
service answers to its name.
Fixes: pymysql.err.OperationalError (2003, "Can't connect to MySQL
server on '127.0.0.1' ([Errno 111] Connection refused)").
Three failures, all from the workflow inheriting whatever the runner
happened to have rather than declaring what it needs.
- backend ran pytest without installing it. pytest is in
requirements-dev.txt, which nothing installed, so the step died with
"No module named pytest".
- Both Python jobs used the runner's system interpreter. The old host
executor had a matching Python by accident; a container image has
3.10, where alembic additionally requires tomli. requirements.txt is
hash-pinned and compiled on 3.14, so tomli has no hash and pip refuses
the whole install under --require-hashes. 3.14 matches the Dockerfile,
the README and the .github workflow.
- The utf8mb4 step shelled out to the mysql client, which is not in
every runner image. pymysql is already a runtime dependency.
The comment explaining the concatenation fix spelled the example hostname out
in full, which tripped the publication scrub - the same trap the surrounding
comment warns about, since the scrub greps this file for the very strings the
patterns are assembled from. Reworded with a neutral example.
Re-verified after the reword by planting a violation and watching the gate fail.
Guides was shipped without an entry in PLUGINS.md or START-HERE, so a site
adopting the product had no way to find out it exists.
BACKUP-KINDS.md described a host's badge as coming from the most recently
confirmed kind. It has not worked that way since the fleet report was reworked:
the freshest revision per (host, kind) is taken, and then the STALEST of those
kinds decides, because reporting the freshest would let a healthy NTLARS backup
mask a part-marker capture that stopped a month ago. Stalest among kinds that
can be JUDGED - a kind never confirmed is unmeasurable, not stale, and treating
it as the worst value made a host with a working backup show no badge at all.
The backups row also still advertised UDC and CMM kinds that do not exist.
api-inventory, openapi.json and PROJECT-MAP regenerated.
Unlike its twenty siblings it ran a bare add_column plus create_index, so
`flask plugin upgrade-all` failed with "duplicate column name: gagelabtag" on
any database whose tables were built by create_all and stamped nowhere - and a
chain stops at its first error, so 0004 never applied either.
The index is guarded by COLUMN SET rather than by name: create_all names indexes
by SQLAlchemy's convention and the migration by this repo's, so a name-keyed
guard would have added a second index over the same column instead of skipping.
The test that should have caught it was restricted to plugins whose whole chain
is a single baseline, on the reasoning that a longer chain meets a second
problem - an ALTER whose column is already present - that guarding a create_table
does not fix. True, but the conclusion was wrong: the answer is to guard the
ALTERs too, not to stop testing the chains that have them. It now covers every
post-cutover chain, and was confirmed to fail against the unguarded migration
before the guard went in.
The NTLARS/DNC registry carries an FTP password in cleartext and it was stored
verbatim. That put it in the backuprevisions table, on the revision panel which
renders values, and in the download route which hands back a ready-to-import
.reg - so anyone holding backups.view could read it off a web page.
Redaction happens at PARSE, so the secret never reaches the database at all.
Redacting on display would have left it at rest and one query away.
What is deliberately NOT done is dropping the value silently. This codec already
refuses to discard an unparseable value, precisely because a backup that looks
complete and restores a machine with a setting missing is the worst outcome. A
redacted value keeps its name, is listed in the projection, and renders as a
COMMENT naming what must be set by hand. Emitting a placeholder as the password
would produce a machine that looks restored and cannot connect, and the
technician would go debugging the network.
scripts/redact_backup_secrets.py scrubs revisions recorded before this, dry run
by default. It leaves contenthash alone: the hash identifies the configuration a
bay reported, so rewriting it would make every bay's next post look like a
change and insert a duplicate revision across the fleet.
Separately, sourcehostname is now normalized at ingest. samesource() maps '' to
IS NULL for the query while the insert stored the raw '', so rows written by a
client posting an empty hostname could never match their own next post: every
post inserted, the table grew without bound, and the history filled with
identical entries nobody could read. Whitespace did the same thing more quietly.
Every domain catches broadly and returns no rows so one broken table cannot take
the whole search box down. That part is right. On its own, though, it makes a
DOWN domain indistinguishable from an EMPTY one: the technician sees no results,
is offered a spelling correction, and concludes the record does not exist. It
does - the search just could not look.
Failed domains are now named in the response and stated above the results.
The suggestion is deliberately still offered on a degraded run. Suppressing it
was tried first and is worse: a chronically unreachable domain, an external HR
directory say, would silently disable spelling help for every search for ever
with nothing saying why. Naming the failure is the honest fix; withholding a
second useful signal is not.
Guides search also moves to the published revision. It correctly limited itself
to published GUIDES, but matched and displayed title/summary/tags/blocksjson
from the Guide row - which is the WORKING copy. A published guide with
unpublished edits matched on wording that only exists in the draft, and the
reader opened it to find the word absent.
The audit log is append-only and nothing ever removed a row, so it grows for as
long as a site runs. This site reached tens of thousands of pages, which is past
the point of being useful: nobody pages through that to find anything.
Retention is OPT-IN and the default keeps everything. An upgrade that silently
started deleting a site's audit history would be indefensible, so
audit_retentiondays defaults to 0 and nothing is removed until somebody sets a
number. The Site & Facility card shows the log's size and exactly how many
entries a purge would delete BEFORE anything is pressed.
Deleting happens in committed batches. That is not an optimisation: one DELETE
over a million rows builds a single enormous transaction and holds locks across
the table while it runs, which on this server means blocking the collectors
posting inventory. The API caps its batch count so the request cannot outlive a
proxy timeout, and a run that stops early reports how much is left rather than
claiming it finished - a purge reporting success over 900k remaining rows would
be the same silent half-failure this codebase keeps getting bitten by.
`flask audit stats` and `flask audit purge` for the server, the latter reporting
by default and deleting only with --apply, with no batch cap because it is not
behind a proxy and a purge that stops halfway leaves the operator guessing.
audit.purge is a separate permission from audit.view. Reading the log and
destroying it are different acts, and plenty of people need only the first.
Chunking selects a page of ids and deletes those rather than using
DELETE ... LIMIT, which is MySQL-only while the suite runs on SQLite.
The patterns are assembled from fragments so the published script does not
contain the strings it greps for. They used the apostrophe-escape form
('a'\''b'), which EMBEDS a literal apostrophe, instead of the concatenation form
('a''b'), which joins them. Every pattern therefore carried a stray quote in the
middle and could never match a line of real code, so the check passed everything
from the day it was written.
Verified the only way that means anything: plant a violation, watch it fail,
revert. Fixed, it immediately found 25 real hits.
Among them, in seven places, the default ServiceNow URLs - one organisation's QA
tenant, shipped with the integration ENABLED. Any site that had not configured
ServiceNow still rendered ticket links, pointing its technicians at a tenant
they cannot reach; an adopting site would have been pointed into somebody else's
instance entirely. There is no sensible cross-site default for "which ServiceNow
is yours", so the URLs ship blank and the integration ships off, matching the
Zabbix block that got this right.
A named SaaS tenant is as site-specific as a hostname, so it joins the patterns.
Colocated .spec.js files are excluded the same way tests/ already is: sample data
naming a facility is not a shipped literal.
Images were a broken icon for every user. The stored URL was bound straight
to <img src>, and a plain image request carries no Authorization header while
this app keeps its JWT in one, so every fetch was a 401. They are now fetched
through the API client, which also fixes the stored '/api/...' path under a
subpath mount where the API is not at /api at all. A failed image says so in
words rather than rendering as a broken icon, and the editor's own thumbnails
go through the same path - an author needs to see the screenshot they pasted.
Revision and image routes gated on their parent guide. Both were gated on
guides.view alone, and revision ids are sequential, so being able to read one
guide was enough to walk the ids and read every unpublished draft on the site -
the opposite of what docs/GUIDES.md promises. An image in a PUBLISHED guide
still serves to any reader; one that only appears in a draft does not.
The guide list, the tag chips and search now match the PUBLISHED text rather
than the working copy. Matching the working copy handed readers a hit on
wording that is not in the page when they open it.
The slug no longer follows the title. It was regenerated on every save that
carried a title and the editor autosaves every three seconds, so fixing a typo
in a heading silently moved the URL and broke exactly the pasted links the
stored slug exists to protect.
publish() now aborts when the pre-publish save failed. It ran regardless, so
the author saw "Published version N" while readers got the previous revision.
Link rendering: text is escaped before formatting, so a captured URL arrives
carrying entities and was escaped a second time - every multi-parameter
ServiceNow and SharePoint link opened with &amp; in it. Emphasis could also
reach inside an href, and '//host' passed as an in-app link when the browser
reads it as off-site.
Tests are now HTTP-level. The draft-invisibility rule lived entirely in routes
that no test executed, which is how three of them shipped serving drafts.
Five of the seven asset types had no way to take a record out of ShopDB from
the interface. Network devices and 3D printed parts had a button; machines,
PCs, printers and measuring tools did not, so a scrapped tool stayed in every
list and report until somebody went to the API.
The endpoints were already there and already correct. Every one of these
DELETEs sets asset.isactive = False and nothing more, so the record leaves the
lists while its calibration history, backups and relationships stay exactly
where they were.
Called "Retire" rather than "Delete" for that reason: the two existing buttons
say Delete, which reads as destruction of a record that is in fact kept and can
be restored by setting isactive back. The confirm says so too, because the
question somebody needs answered before clicking is whether the history goes
with it.
The button is guarded on the record having loaded, on the two pages whose
action row renders before the fetch resolves - measuring tools showed Retire
beside a hidden Edit while the tool was still loading, which would have called
delete with an undefined id.
printersApi had no delete method at all; added.
USB is deliberately left out. It retires through a status column in the
external CMMC database rather than isactive, has its own /retire endpoint, and
needs its own decision about what retiring means there.
Every detail page read "West Jefferson ShopDB - Machine". Two tabs on two
different machines were indistinguishable, a bookmark said nothing about what
it pointed at, and a link shared into Teams previewed as the section rather
than the thing.
The router cannot fix this alone: afterEach runs before the view has fetched
anything, so it only ever knows the KIND of page. So a detail view names its
record once the record arrives, and both writers go through
composables/pageTitle.js.
THE ORDERING IS THE WHOLE PROBLEM. The router awaits the site settings to build
the "<facility> ShopDB" prefix, so on a cold load it finishes AFTER a view that
fetched quickly and would overwrite the good title with the generic one. The
route's fullPath is claimed before that await, and the generic write stands
down when a record title already belongs to the route being landed on. The
reverse case is handled too: a view that names its record BEFORE the facility
name arrives composed against the placeholder prefix, so the late resolve
rewrites it rather than leaving "ShopDB - CSF16" on a site called West
Jefferson. Both are pinned by tests, because both fail only under timing.
Covers machines, PCs, printers, network devices, subnets, measuring tools, USB
devices, printed parts, applications, employees, knowledge base articles and
guides. Printers reuse the page's own displayTitle, which already discards the
literal name "NONE" a discovered queue can carry, so the tab and the heading
agree.
A record with no usable name, or one that failed to load, keeps the section
name: "Machine" beats "Machine 4192" for somebody scanning a row of tabs.
Long names are cut at a word boundary - knowledge base articles are named with
a whole sentence of description, which a browser otherwise truncates mid-word.
Reports gained meta.title, which the router already supported and nothing used.
That also fixes "Pc Relationships", which the name-derived spelling produced.
Two faults the guides plugin shipped with, both caught by gates that were
already there.
deploy/site-profile-universal.json never listed `guides`. That file is the
profile the released Windows installer is built from, so the wizard would not
have offered Guides at all - and worse, `flask plugin prune-schema` drops the
tables of any plugin absent from the profile at provisioning (ADR-014). A site
installing the next release would have had its guides table dropped.
plugins/guides/models/guide.py imported shopdb.core.models.base directly.
Plugins reach core only through shopdb.api or shopdb.plugins.base (ADR-001),
and shopdb.api already re-exports BaseModel, so this is the same symbol by the
supported path.
The suite was red on main because of these two; it passes again.
Global search has per-domain toggles (search_<type>_enabled), enforced
server-side and edited from Settings. Guides answered the search box but had no
toggle, so it was the one domain nobody could turn off - and the omission would
only have surfaced when somebody went looking for the switch and could not find
it.
Missing key still means enabled, so nothing changes until somebody decides
otherwise.
The repo already enforces that every served route appears in
docs/api-inventory.json, and the guides endpoints did not - so the parity test
was failing. They are now listed with auth, params and a working example each,
beside knowledgebase because the two get read for the same reason.
docs/GUIDES.md covers what the inventory cannot: the block schema and why it is
blocks rather than HTML, the id rule that makes a tick rememberable, what the
server refuses and why validation happens at the door, the working-copy /
revision / published-pointer split that stops a live procedure changing under
somebody following it, and a worked example of creating and publishing a guide
from a script.
It also states plainly what does not exist yet - no importer, orphaned images
never pruned - and what converting existing documents will actually be like:
structured HTML and Markdown map onto block types nearly one for one, while a
plain-text runbook lands as paragraphs and needs marking up by hand afterwards.
Generated ids have to stay stable across re-imports, or every re-import resets
readers' progress.
Guides moves to position 8 in the sidebar, beside Slides, rather than down among
the reference lists in Information. A guide is something somebody goes and does
something with, not a catalogue they look things up in.
A guide was invisible to global search. Core search has a hardcoded block per
domain rather than a hook, so a new plugin stays unreachable until someone adds
one - which is the same shape as the gap this test file already documents for
USB devices and printed parts: records with a list page, labels in the world,
and no way to find them from the search box.
Guides are searched by title, summary, tags AND body. The body matters more here
than elsewhere: people look for a procedure by something written halfway down it,
not by a title they never read. PUBLISHED ONLY - a draft is deliberately
invisible to readers, and search is exactly where that rule leaks if nobody
writes it down. The test proves it with a draft sharing a tag and a body word
with a published guide, so a missing filter would show up as a ride-along.
PROOFING catches what a spellchecker cannot, because none of these are spelling:
a smart quote in a command so it will not run, a checklist item with no text so
a step is silently missing, a heading left with nothing under it after a
reorganise, an image with no alt text, a TODO published by accident. Warnings,
never blocks - somebody publishing at 14:40 with one empty bullet should be
told, not stopped, and a procedure that exists imperfectly beats one nobody
wrote.
The smart-character issue reports once per block with a one-click fix, rather
than once per offending character: an opening and a closing quote are one
problem, and listing both makes the panel look like it is nagging.
The character table itself is written as escapes. The style gate forbids
non-ASCII in source, and a file whose job is detecting smart quotes cannot
contain any - which the gate caught, correctly.
Seven changes, all aimed at the same thing: think about the procedure, not
about blocks.
Live preview beside the editor, rendered with the SAME component readers get,
so what an author sees is what will be published - a separate preview
implementation would drift within a month.
Formatting buttons wrap the selection, so nobody needs to know the markup is
Markdown. New blocks and pasted images land AFTER the block being worked on
rather than at the bottom of the document, which removes the scroll-back that
made adding a screenshot mid-guide annoying.
Enter in a list makes the next item and focuses it, the way every other list in
every other tool behaves; backspace on an already-empty item removes it and
steps back up. Typing a ten-item checklist no longer needs the mouse.
Templates, because the blank page is the real barrier to somebody writing a
procedure down at all. The shapes are lifted from the KB articles the team
already writes, so guides written here read like the ones written before them.
Autosave on a three-second debounce with an honest status line, safe precisely
because the server dedups on content - an autosave that changed nothing writes
no version. Leaving with unsaved work now asks first.
CLIPBOARD: the code block no longer writes to the clipboard. A page that
programmatically copies a shell command is the exact shape of a ClickFix attack,
where malware plants a command in your clipboard and talks you into pasting it
into Run - and uBlock Origin blocked ours, correctly, because it was writing
PowerShell on click. The button now SELECTS the code and says to press Ctrl+C:
same convenience, decision back with the person, and nothing for security
tooling to object to.
AUTOCORRECT is on for prose and off for code and table cells. A code block that
autocorrects produces a guide telling somebody to run a command that does not
work - smart quotes replacing a straight one, an autocapitalised cmdlet - and it
still LOOKS right to the author. The person at the bay gets the syntax error.
Printing a dark-mode page lays a full page of toner down for the background and
knocks the text out of it in white. On a shopfloor printer that is a cartridge
for a handful of guides, and the result is harder to read than the plain
version. Somebody would have found this the expensive way.
The theme tokens are redefined for print only, in an unscoped block because
overriding them has to reach :root. Every guide element already renders through
those variables rather than literal colours, so this is a dozen lines instead of
a rewrite - the same discipline that made dark mode work without dark-mode code.
Backgrounds are dropped rather than printed grey: a callout is identified by its
rule and its heading, which cost nothing. Checkboxes print EMPTY even where they
are ticked on screen, because a printed guide is a worksheet somebody carries to
the bay, and one person's progress is not part of the procedure. The footer
gains the print date beside the version, so a stale copy pinned to a noticeboard
can be spotted.
Also corrects the model and migration found by running this against a real
database rather than SQLite: BaseModel contributes createddate, modifieddate and
isactive to every table and the migration never created them, so the first read
failed on unknown columns; the model had also declared duplicate timestamps on
top, now removed with the API field names kept stable. And the slug is 191
characters, not 300 - it is UNIQUE-indexed, MySQL caps an index key at 767
bytes, and utf8mb4 is 4 bytes per character.
The knowledgebase plugin is a link registry - a description, a URL and some
keywords pointing at a document held somewhere else. That leaves the documents
themselves as HTML files on a share: no search, no history, no way to know
whether the copy pinned to a noticeboard is current, and no way for a technician
to tick off where they are.
A guide is stored as an ordered list of typed blocks rather than a blob of
HTML, because three of the four things this feature exists for depend on it. A
tickable step needs a stable id to be remembered; dark mode needs semantic
markup rather than pasted colours; and print needs to know where a step begins
so it does not split one across a page. Free HTML gives none of that and lets
pasted styling fight the theme.
Inline text is a restricted Markdown subset - bold, italic, inline code, links,
line breaks - escaped BEFORE formatting, so nothing an author pasted can reach
the DOM as markup. No fonts, sizes or colours: emphasis is semantic and the
theme decides how it looks. That is what makes a guide written next year match
one written today, and dark mode work without an author thinking about it.
DRAFTS AND VERSIONS. The guide row is the working copy; every save that changes
something writes an immutable revision; and a separate pointer says which
revision READERS get. Without that pointer, editing a live procedure changes it
under a technician who is halfway through following it. Publishing promotes an
existing revision rather than snapshotting, so a publish with unsaved edits
cannot ship a version nobody reviewed. Restoring writes a new revision with old
content, so the record of what people were told in between survives.
Saves dedup on a content hash covering title, summary, tags and blocks, so an
autosave that changed nothing writes no version - otherwise the history fills
with identical entries and hides the three edits that mattered.
Progress lives in the browser, not the database: shopfloor PCs share a login, so
per-user ticks would show one technician's progress to the next person on that
bay, which reads as "already done". Keyed by guide and version, because step 4
of version 3 is not step 4 of version 2.
Five permissions, and the split between edit and publish is the point: a
technician can write a procedure up while somebody else decides it is fit for
the floor. create-without-edit works too - an author may edit their own guide
until it is published, after which other people are following it.
CI caught this: judging a host by its stalest kind treated a never-confirmed
revision (lastseenat NULL) as the worst case, so it won. The verdict for a NULL
confirmation is tri-state None, which renders as NO BADGE - so a host with a
healthy ntlars backup and one backfilled row showed nothing at all. Hiding a
working backup is the one thing that badge must never do.
A NULL confirmation is not stale, it is unmeasurable. The stalest kind is now
chosen among kinds that can actually be judged, which is the rule staleness.py
already applies: a chain becomes measurable the first time its PC posts and says
nothing until then. If every kind is unmeasurable the host still reports which
kind it has, with a None verdict and no badge.
The test that caught it pinned the old newest-wins behaviour, which part markers
made wrong; it now pins stalest-measurable-wins and keeps its NULL clause, which
was right all along and is what found the bug.
Also worth recording how it was missed: the change was verified against
test_geenforce_reporting.py and test_backups.py, and this assertion lives in
test_geenforce_publish_gate.py, which was never run.
The publication scrub gate rejects internal references, and a real site
hostname in a test is still one in the published repo. The rewrite does not
care which server it is.
Snapshots written before the client was fixed stored "W:\backup\partmarker\..."
as their location. That mapping exists only inside the SYSTEM logon session,
only during an enforcement cycle, and only on the bay that wrote it, so the path
recorded against a marker resolved to nothing for the person most likely to open
it.
The script rewrites backuprevisions.sharepath and nothing else - no revision is
created, deleted, re-filed or re-hashed, and no file on the share moves, since
the bytes are already where the UNC will point. Dry run by default.
It matches on the drive letter alone and leaves everything else untouched,
including paths that are already UNC and paths on another drive. This repairs a
specific known mistake rather than normalising whatever it can reach, which is
the version of this script that would eventually rewrite something it should
not have.
rewrite() is kept pure so the decision of WHICH paths to touch is testable
without an app context; the tests cover the already-UNC, other-drive, empty,
case-difference and trailing-separator cases.
The plugin shipped five settings and no page for any of them. The core settings
catalog is a hardcoded list a plugin cannot extend, so backups_shareroot and the
collection interval were reachable only through the API or a flask shell on the
server - which is not a configurable setting in any useful sense, and it is how
today went: the setting simply was not there to set.
The page is contributed the ADR-010 way rather than by editing core - a card
from get_settings_cards plus a route in the plugin's own routes.js, both of
which disappear when the plugin is disabled. Filed under Machines, next to
Machine Types, because that is where the assets these back up live.
It also states the two things the settings do not say for themselves: that
health is judged on when a backup was last CONFIRMED rather than when it last
changed, and that retention prunes database revisions only - files on the share
stay, so enabling it on a share-backed kind leaves orphaned snapshot
directories.
Snapshot paths are now shown as full UNC text in both places: a Snapshot row on
the Marker Contents card, and the existing path on the history page, which
gains a copy button. Deliberately NOT a link - browsers refuse to navigate from
an https page to file://, so a clickable path looks actionable and silently
does nothing, which is worse than no link. Copy, then paste into Explorer.
The path comes off the revision, never composed from the configured share root:
a snapshot written before the layout changed still lives where it was put.
Until part markers, a PC reported exactly one backup kind and "newest revision
wins" was the same thing as the truth. A marker bay reports two: ntlars for the
machine's DNC settings, partmarker for the controller's patterns.
With newest-wins, the daily ntlars post keeps the badge green while the marker
backup has been failing for a month. The failure shows only on that marker's
asset page, which is not where anyone looks to find out something stopped - and
"is this still being checked" is the one question this badge exists to answer.
So a host is green only when everything it backs up is current, and the badge
names the kind that is not. Freshest revision per kind first, because
lastseenat only ever moves on the newest revision of a chain; judging every row
would report a healthy, frequently changing config as permanently stale.
Known consequence, and the right direction to err: a device retired from a PC
leaves its last revision behind, so that host stays stale until the revision is
pruned. It complains about a backup that genuinely is not running rather than
staying quiet about one.
The dashboard card needed no change - it already lists every stale chain
independently, so a stale partmarker chain was never hidden there.
A 'share' kind keeps its bytes on the file share and gives ShopDB only a hash
and a UNC path. That is right for an opaque vendor file, but it means the app
cannot answer the question a tech actually arrives with - what does this part
marker hold, and which pattern disappeared last month. Reading the share from
the app server is not an option: the download routes refuse to mount it on
purpose, because a permissions slip there becomes an unexplained empty panel.
So the collector now accepts an optional manifest alongside the hash: the
index, not the content. Names, sizes, per-file hashes, and the marking system
each pattern was authored for. Advisory throughout - a client that sends none
behaves exactly as before, and dedup still keys on contenthash.
The marker asset gains a Marker Contents card built from it: counts by type,
the controller family, and a diff against the previous snapshot in the same
source chain. Removed files are named rather than counted, because "1 removed"
only tells someone to go and look. Files edited in place are named too - the
file set can be identical while the content hash moved, and reporting no
change there would contradict the reason the revision exists.
The manifest is deliberately NOT covered by contenthash, which attests the
bytes on the share. A client that starts describing itself therefore posts an
unchanged hash, so a no-op backfills a manifest the row does not yet have -
otherwise the card would appear only on markers that had recently changed, and
stay hidden on every healthy one. The same no-op refreshes sharepath, so a
snapshot that moved on the share stops being pointed at where it used to be.
Manifests are validated at ingest rather than at render: a malformed one would
otherwise become the latest revision and fail every view of the asset page
until the content next changed, outliving the client bug that caused it.
sharedir() for the marker kind is overridden to the layout the site actually
uses, so what ShopDB displays and validates matches what the collector writes.
Canonical names chosen by the site: Dell Inc., HP, Zebra. The rule is the same
in each case - keep the spelling that holds the entries - it just lands on a
different form per company. Dell Inc. carries 310 references and 33 models
against 1 apiece for Dell and DellInc.; HP carries 51 and 24 while HP Inc. has
none; Zebra carries 7 and 2 while Zebra Technologies has none.
So HP Inc. and Zebra Technologies are gone from the shipped catalog. Both were
empty rows that only ever offered a second way to spell a vendor already
present, and neither exists in this site's database at all - they would have
arrived as duplicates at a NEW site and nowhere else.
Also removed UnscopedLiveVendor, which is a test artifact. It reached the
catalog because export_catalog.py reads the live development database, so a row
left behind by a lab run was published as reference data and has shipped to
every site seeded since.
Vendors 51 to 48. Verified nothing referenced any of the three before removing
them, and the catalog still has no duplicates by any spelling.
The seed catalog is what a new site inherits, so its duplicates propagate.
Removed the four space-stripped twins it carried: vendor 'DellInc.', model
'OptiPlex7080', and operating systems 'MicrosoftWindows10Enterprise' and
'Windows10', each of which had a correctly spaced original already present.
'OptiPlex5000' was NOT a duplicate - no spaced twin existed - so it was
renamed to 'OptiPlex 5000' rather than dropped, matching every sibling in the
list. Its vendor moved to 'Dell Inc.', which this site treats as the one
canonical Dell.
Vendors 53 to 51, models 128 to 127, operating systems 14 to 12. Verified
afterwards: no exact duplicate rows, no near-duplicates, no orphaned
model-to-vendor or supply-to-model reference.
The legacy data carried space-stripped twins: 'DellInc.' beside 'Dell Inc.',
'Windows10' beside 'Windows 10', 'OptiPlex7080' beside 'OptiPlex 7080'. They
rode into the shipped seed catalog, so every site that ran `seed catalog`
inherited them, and they stay invisible until somebody opens a vendor picker
and finds the same company listed twice.
Not a code fault. The collector strips only leading and trailing whitespace,
matches with ilike, and treats the OS list as lookup-only, so it can neither
have created these nor be creating more. They arrived with the classic-ASP
import, which holds the same three pairs today.
`flask catalog dedupe` merges only whitespace-and-punctuation twins, because
those are one name typed two ways. 'Dell' and 'Dell Inc.' normalise apart and
are left alone: that is a naming choice and only a person can make it. The
surviving row is the most referenced, tie-broken toward the spaced spelling,
which is the one somebody pressed the space bar for.
`flask catalog merge` makes that judgment call explicitly, and refuses to
invent a target that does not exist, so a typo cannot move rows into a new
vendor. `flask catalog rename` handles the case neither can reach: a stripped
name with NO twin, where there is nothing to merge into.
All three report and change nothing unless --apply is passed, and dedupe is
idempotent so it is safe in a deploy script.
References are discovered from the LIVE schema rather than listed in code.
There are 15 foreign keys into these three tables across 7 tables, and
machines.controllervendorid and printerdrivers.modelnumberid are not ones
anybody would enumerate by hand. Every reference is repointed before the row
is deleted, so nothing is orphaned, and a plugin adding a table is covered
without editing this command.
Running `python scripts/export_catalog.py` on a deployed site died with
1045 "Access denied for user 'root'@'localhost'", naming a user that site
does not have.
Nothing loads .env for a plain script run. wsgi.py calls load_dotenv() for the
served app and the flask CLI loads .env for itself, so both reach the real
database; a bare `python scripts/...` gets neither. DATABASE_URL then stays
unset, FLASK_ENV defaults to development, and config.py falls back to a
hardcoded root@localhost URL.
The read-only failure above is the harmless version. Six of these seven
scripts WRITE - backfill, reclassify, retype, collapse, refile, import - and
on any box where those fallback root credentials happen to resolve, they would
have silently rewritten the wrong database with no error at all. That is also
why this survived: the dev .env points at root, so the fallback matched what
the developer expected and nothing ever looked wrong here.
All seven now load .env from the repo root, resolved from __file__ rather than
the working directory, since these are run by hand during a deployment and
rarely from the repo root.
tests/test_maintenance_scripts.py pins it for any future script: every script
that calls create_app must call load_dotenv, before create_app rather than
after it, and with an explicit path rather than a cwd-relative search.
PLUGINS.md listed 14 of the 15 bundled plugins. `backups` had no row: the
per-asset configuration backups the collector takes off shopfloor machines,
kind-pluggable, with view and download as separate permissions because a .reg
is a restorable artifact. Added, with the pointer that separates it from
BACKUP-RESTORE.md, which is the database and a different subject.
PLUGIN-LAB-PRINTEDPARTS said the finished implementation lives on branch
feat/printedparts-plugin. That branch was deleted in the v0.8.0 cleanup; only
the 17 lab-stage tags survived, and they are enough. Says so now, and warns
that the tags are a record of contract 0.11-0.13 rather than today's
conventions. Its manifest example carried core_version >=0.12.0 against a
shipped >=0.16.0.
The printer-assignment proposal still said the client script is not built. It
shipped and released, across v0.11.3 and v0.12.0, and the collector half that
contradicted the assigned-vs-observed split was retired in adc7ef4.
PROJECT-MAP regenerated for the test count.
`flask plugin new` wrote views to frontend/src/views/<name>/ and a route file
to frontend/src/router/routes/<name>.js. Every one of the 16 bundled plugins
keeps its frontend in plugins/<name>/frontend/ instead, staged into
.plugins-staged by stage-frontend.mjs and aggregated into routes.gen.js.
It was not visibly broken, which is why it lasted: the router still globs
./routes/*.js during the Phase 4 dual-location transition, so a scaffolded
plugin's pages loaded. What a new plugin author silently lost was ADR-013 lean
builds. Pruning works off the staged tree, so a frontend in the core tree
cannot be excluded from a site build, and deleting the plugin strands its
views behind.
The templates for the correct layout already existed; only the output paths
were wrong. Also fixed the imports they carry: core imports now use the @
alias (@/api, @/components/...), which resolves both before and after staging,
where the previous ../../api was correct only in the old location. Sibling
views stay relative and travel with the route file.
frontend_dir is gone from scaffold_plugin. There is no second output tree to
point at, and a plugin developed in its own repo now gets the same frontend as
a bundled one rather than being skipped.
Proven end to end: scaffolded a throwaway plugin, confirmed it stages, builds
real chunks, and that SITE_PLUGINS=network prunes it, which is what the old
layout could not do.
A NOTICE result linked to /notifications, so clicking the row search had
already identified dropped the reader on the list page to find it again by
hand. The id was in the payload the whole time; only the URL ignored it.
`/notifications/<id>` is the existing notification-detail route.
The subnet searcher had the same defect two functions away: an IP search
resolves which subnet the address falls inside, then linked to /network, the
hub. It now links to /networks/<id>, the subnet detail route.
Every other search result type already carried its id. These were the only
two that did not.
The printer feature rests on one rule: a collector report is OBSERVED state and
never ASSIGNED state. If a drifted bay's own state can become what it is told to
install, enforcement stops meaning anything, because a bay that installed the
wrong printer makes itself right simply by reporting it.
_sync_printer_links broke that rule. It ran on every collector PC push and, when
the payload carried defaultprinter or printers, wrote defaultprinter and
connectedto rows from the report - defaultprinter being one of the two types
_reconcile_edges is meant to own alone, from authenticated routes a human calls.
It was recorded as dormant. It was not. The collector schema ADVERTISED both
keys and the example client in COLLECTOR-INTEGRATION.md set
$payload['defaultprinter'], so the documentation was recruiting callers for it -
while the same document, under "Observed is not assigned", promised that nothing
on this path writes a usesprinter or defaultprinter row. The document
contradicted itself and the code followed the wrong half. Nothing in the fleet
armed it, because the first-party reporter never sent those fields, but any
third-party client written from the docs would have.
Retired the function, its _sync_one helper and the now-unused
PRINTER_LINK_ORIGIN label, and removed both keys from the schema.
IGNORED, NOT REJECTED. Unknown payload keys are not validated, so a client
written against the old docs keeps pushing successfully and only its printer
keys stop doing anything. The response warns, naming the keys and pointing at
POST /api/collector/printers for reporting queues and seed-from-observed for
adopting them. Dropping them in silence would leave an integrator believing
assignments were still being written, which is a worse failure than the one
being fixed.
The docs were the larger half of this. The schema table rows are gone, the
"PC -> printer relationship sync" section is replaced by a note saying what
happens now to anyone still sending the keys, and the example client no longer
enumerates Win32_Printer at all - it points at Report-PrintersToShopDB.ps1,
which posts to the observed endpoint and posts NOTHING on a failed enumeration,
since an empty list would erase the host's rows.
test_collector_printers.py pinned the old behaviour, so it now pins the
opposite: the keys create no relationships, the push still succeeds and says
why, a payload without them stays quiet, a human-made default survives a report
naming a different printer, and the schema no longer offers the keys.
Reported: the warranty page cannot be searched by machine number.
It was there to be found. _related_machine puts the machine's number in the
payload, and it does so for a stated reason - a shopfloor PC is bought,
warranted and replaced as a PC, but it is FOUND by the machine it drives, and
nobody walks the floor looking for an asset number. The table renders it as a
chip. The filter looked at vendor, service level, service tag and asset number,
and never at the machine. So the one identifier the feature was designed around
was the one thing the search box could not match.
This is how a field list goes wrong: a column was added to the table and not to
the filter, and nothing connects the two. Moved the field list into
warrantySearch.js so a test can see it, following controllingPc.js and
labelPages.js. Machine name is searched as well - somebody who knows the bay by
"the Makino" should not have to look up its number to find its warranty.
The specs also pin the case that made the old inline version fragile: a covered
asset whose machine is null, which is every printer and any PC nobody has linked
yet, must not throw. And a missing field must not match: String(undefined) is
'undefined', so a naive coercion makes a search for "und" hit every row with an
empty vendor.
Not changed, worth knowing: the covered asset's NAME is still not searched, only
its number. That looks like the same omission, but it is a wider change than the
report - the placeholder has always said "asset" without saying which half - so
it is left for a deliberate decision rather than folded in here.
Reported: `flask plugin upgrade-all` failed the tools plugin with "table
securesendsecrets already exists".
Every post-cutover baseline emitted a plain op.create_table. That is correct
only while a table's existence and its plugin's Alembic stamp can never
disagree, and they can. Anything that builds tables from the models -
db.create_all(), which `flask db-utils create-all` calls - creates them and
stamps nothing. A run that creates a table and then fails before its stamp
commits leaves the same split, and re-running is the obvious thing to try. In
both cases upgrade-all meets existing tables with an unstamped chain and the
create is fatal.
Nothing caught it because the migration test DROPS the post-cutover tables
db.create_all() built before running upgrade-all. That arranges the one state
where an unguarded create works, and never the state the report came from. The
new test is the same sequence with the drop removed for the plugin under test.
create_table_if_missing and create_index_if_missing are in alembic_template.py
next to create_plugin_tables, which had the idempotency rule right and turns out
to be called by nobody - its docstring claimed every 0001 used it. Fixed the
docstring rather than rewiring the baselines onto it, because the reason they
hand-write their DDL still stands: the helper's per-plugin MetaData cannot
resolve a foreign key to a core table at CreateTable-compile time.
INDEXES ARE MATCHED ON COLUMNS, NOT NAME, and that distinction matters. The two
creators disagree about names - db.create_all() takes SQLAlchemy's default
convention (ix_securesendsecrets_token) and a baseline spells it the way this
repo's naming rules require (ixsecuresendsecretstoken). A name-keyed guard would
have called the index missing and added a second one over the same column, which
MySQL permits with a warning, so every affected site would have quietly grown
duplicates. The test asserts no table ends up with two indexes over the same
columns.
Applied to all nine table-creating plugin migrations, not just the tools
baseline that was reported: the four later migrations that create tables have
the same exposure, and geenforce, measuringtools, printedparts and backups are
the same defect waiting for a different site.
WHAT THIS DOES NOT FIX. It makes a baseline idempotent, not a whole chain. A
longer chain can still meet an ALTER whose column is already present -
printedparts adding gagelabtag to a table create_all built at head is the case
that showed up while testing. Guarding every op one at a time is the wrong
answer to that; a database built from the models is already AT head and its
chains want stamping rather than replaying, and there is no `flask plugin stamp`
today. Left alone deliberately rather than half-built, and the new test is
scoped to single-baseline plugins so it does not pretend otherwise.
Reported: moving machine 3016 on the map and saving answered "this
relationship already exists" and refused the save.
The map had nothing to do with it. Every save of the machine form re-POSTed the
controlling-PC link, and the server was right to refuse it - the row was already
active, and assetrelationships is unique on (source, target, type). Any edit to
a machine that has a controlling PC hit this; a position change is just the one
somebody happened to be making.
The form asked the wrong question. It used "do I know the existing link's id" as
a stand-in for "is there an existing link", and read that id from
rel.assetrelationshipid - a key the API has never emitted. to_dict serialises
column names and the column is relationshipid, so the read was undefined on
every load while controllingPcId was populated from the same relationship. The
two answers disagreed, and the disagreement was invisible: undefined is not an
error, it is just a falsy id.
That single conflation caused both failures. Saving anything re-created the
link, and CLEARING the controlling PC did nothing at all, because "no selection
and no known id" reads as "nothing to do" - so removing a machine's controlling
PC silently kept it. Only the first of those was reported, and the second is the
one somebody would have trusted.
Fixed by making the LOADED PC the authority on whether a link exists, with the
relationship id used only to delete a row. If that id ever goes missing again
the worst case is a link that cannot be removed, never a duplicate create.
The decision moved to controllingPc.js as a pure function so it could be pinned
by tests, following labelPages.js. Unchanged is now explicitly no-op rather than
delete-then-create: re-linking a machine to the same PC opens a window where it
has no controlling PC at all, and resolve_asset_position walks that edge, so a
failure between the two calls would drop the machine's inherited position along
with its link. The specs also cover a select handing back '42' against an API
that gave 42, which would make an untouched link look like a swap and reach the
same duplicate POST by another route.
Not the printer assignment, which was the first guess: PrinterAssignmentPicker
makes no relationship calls, and its save is wrapped in a catch that only logs,
so it cannot surface an error from this form at all.
Secure Send shipped on 21 August and nothing outside the code caught up. The
roadmap still listed it under Deferred as "tabled", and gave as the reason that
the tools plugin is browser-only by design with get_blueprint and get_models
both empty - which stopped being true in the same day's commits that gave it a
blueprint, a table, a migration chain, a permission and a setting. A reader
following the roadmap would have concluded the feature does not exist.
The generated API docs were fine: api-inventory.json and openapi.json both
carry /api/tools/secrets, so the contract gates did their job. What was missing
was every word written for a person, and that is the half where the security
properties live.
docs/SECURE-SEND.md is the new page. The parts worth having in prose rather
than inferring from the code: that a lost secret is lost to everybody including
an administrator, which is the same property as "a compromised database yields
nothing" seen from the other side; that the 4096-character limit is the entire
server-side enforcement, because a server that cannot read what it holds cannot
have an opinion about content, so raising it turns a password tool into an
unaudited file transfer; that redeeming is a POST because preview prefetch in
chat and mail clients would otherwise spend a one-view secret before the
recipient ever sees it; and that somebody should send a link to themselves
through their own mail or chat client and confirm the key survives after the #
before a real credential goes through it, because a product that rewrites links
can strip the fragment and nothing on this side can help.
securesend_allow_guests was in the database, on a settings page, and in no
document. It is now in CONFIG.md with what it actually does - an unauthenticated
write endpoint, rate limited per address, failing closed - rather than only its
one-line description.
Also corrected along the way: site_base_url was documented as scheme + host
while the code deliberately tolerates a subpath in it, and the bundled plugin
table in PLUGINS.md had no row for tools at all.
Still missing after this: PLUGINS.md has no row for backups either.
Reported: no Tools section in Settings. The setting existed in the database and
had no page, which from a person's side is indistinguishable from the feature
not existing.
get_settings_defaults SEEDS a setting. get_settings_cards (ADR-010) is what puts
it in the rail. Declaring only the first is a silent half-feature: nothing fails,
no test notices, and the switch is unreachable.
The page states the cost where the DECISION is made rather than in a document
nobody opens - turn it on and it says a guest-created link records no creator,
so the audit trail can say when and from where but never who.
Admin-only, like every other settings page, because this one decides whether an
unauthenticated caller may write to the site.
Also caught while wiring it: the card asked for a 'shield' icon, which is not in
the settings iconMap, so it would have rendered the fallback puzzle piece - the
glyph that reads as "nobody chose this". Uses wrench, matching Tech Tools.
The row still comes from `flask plugin upgrade-all`; the card appears either
way, and without the row the page loads empty.
/tools, /tools/codes and /tools/secure-send were all behind requiresAuth. None
of them needed to be, and one of them actively contradicted a setting added an
hour earlier: securesend_allow_guests lets somebody with no account create a
link, and the page they would create it on was behind a login wall.
The label generator runs entirely in the browser - no API, no data, nothing to
protect - so gating it only ever cost a guest the use of it.
THE SERVER STILL DECIDES who may create a secret. A signed-in user needs
tools.secrets; a guest needs the site to have opened guest creation. Making the
page reachable is not the same as making the action allowed, and the two were
conflated.
The page now reads the setting on load, so a guest at a site that has NOT opened
guest creation is told to sign in instead of being handed a form that was always
going to be refused - with a line saying that receiving a link never needs an
account, because somebody who lands there confused usually has a link they
should be opening instead.
A signed-in user always sees the form, even without the permission: the server's
"you do not have permission" is a different answer from "sign in", and
collapsing them would send somebody to a login page they are already past.
Two asks, and the first is a real security decision so it is a SETTING that
defaults to the safe answer rather than a code change somebody inherits.
securesend_allow_guests, OFF by default. With it off the model is unchanged: a
login plus tools.secrets. With it on, somebody with no account - at a shared bay
terminal, a contractor - can create a link too.
The cost is named in the setting's own description and pinned by a test: a guest
secret has NO CREATOR, so the audit trail can say when and from where but never
who. That is the whole reason it is a switch a person turns on deliberately.
IT FAILS CLOSED. A missing row, a half-migrated site, an unhappy database - the
answer is no. An error must never be the thing that opens an unauthenticated
write endpoint.
A SIGNED-IN USER STILL NEEDS THE PERMISSION even when guests are allowed, which
is surprising enough that a test pins it: opening the door to guests is a
decision about the door, not about revoking what a role holds.
Guest creation is rate limited per address, same shape as the login limiter. Not
a security boundary - a determined person has more addresses - but an open
endpoint should not be an easy place to park data or fill a disk.
VIEWS ARE NOW A NUMBER, 1 to 100, instead of a four-item dropdown. How many
people need a password is the sender's business: a shift handover is not one
colleague on a call. The floor is what matters, not the ceiling - unlimited
views would just be a password on a web page.
Also folded in, from the retention question: nothing removed a secret that
EXPIRED WITHOUT BEING OPENED. The recipient goes on holiday, the link is never
clicked, and the ciphertext sat there for good - not readable, but it made
"nothing lingers" untrue and grew the table in exactly the case where the
feature failed. Any create now sweeps expired rows and a late click tidies its
own, so it stays bounded without a scheduler an air-gapped site would never add.
Two gates caught mistakes on the way: current_user is a LocalProxy and is never
None, so `current_user is None` silently took the signed-in branch for a guest;
and cache was imported straight from shopdb.extensions when it is already on the
shopdb.api contract surface.
1836 tests green.
Answering "did I set it wrong": no. The setting was right and the description
was misleading, which is what sent the last two commits chasing the wrong fix.
It said "Public base URL of this site (scheme + host), e.g.
https://shopdb.example.net". That is true only of a site mounted at the root.
Both existing consumers append the in-app route to this value and add nothing
themselves:
`${await getSiteBaseUrl()}${config.detailPath(detailId)}` asset labels
`${baseUrl}${defaultPath}` QR codes
so a site served at /shopdb must carry that here or every QR code it prints
404s. The email copy of the same setting has always documented it correctly, as
https://host/ops.
The description now says so, and the comment above the defaults records why -
including that a subpath left out produces links that fail rather than anything
that looks like a configuration error.
Existing rows keep the old text until a site reseeds; the value is unaffected.
a381174 fixed a link missing its subpath and produced one carrying it twice:
https://<host>/shopdb/shopdb/s/<token>.
site_base_url is DOCUMENTED as scheme + host, and sites really do set the full
base including the subpath. Trusting the description rather than tolerating what
is configured is the same mistake as the last one, pointed the other way.
The mount is now appended only when the configured URL does not already end with
it, which covers every combination a site can have: base with the subpath, base
without it, a trailing slash on either, and a site mounted at the root where
there is no subpath at all.
Checked all five by evaluating the expression rather than reasoning about it.
Reported from production: clicking a secret link gave an IIS 404, and the link
read /s/<token> where the site lives at /shopdb/s/<token>.
The link was assembled from window.location.origin, which is scheme and host
only. site_base_url already exists for this and says so in its own description -
"used for QR codes and any absolute link the app emits" - with getSiteBaseUrl()
as the helper the QR pages use. This reached past all of it.
Both halves were wrong, and the second is worse because it fails silently:
MOUNT PATH. A site served under /shopdb or /ops carries that in front of
every route. Leaving it off is the reported 404.
HOST. Whoever creates a secret may have reached the app by an IP, a short
hostname, or a name that resolves in one VLAN, and the recipient opens the
link from somewhere else entirely. The configured site URL is the one address
known to work for both of them.
The shape of this failure is what makes it worth spelling out: the secret was
created SUCCESSFULLY and the sender was handed a link that could never work, so
the failure landed on the recipient, the sender had no reason to doubt it, and
the secret expired unread. A 500 would have been kinder.
Falls back to the browsing origin when a site has not set site_base_url, which
is what every other absolute link here does.
BASE_URL is injected at build time, so the subpath half of this cannot show up
in dev and only appears on a subpath deployment.
The backend has been there since d4b49b9 with nothing to click. This is the
half a person actually touches.
A card under a new "Secrets & Access" heading on Tech Tools, which is where
somebody looks for a utility - the whole reason the plugin moved there.
THE CREATE PAGE encrypts before anything leaves the browser and shows the link
once. The plaintext is dropped from memory as soon as it is sealed, and the page
says plainly that the key is in the link and exists nowhere else, because
somebody who does not know that will paste the link somewhere durable.
THE READ PAGE IS PUBLIC, at /s/<token>, outside the app shell: the recipient
usually has no account here, so it carries no navigation and asks for no login.
Short path because the whole link gets pasted into a chat message.
IT NEVER REVEALS ON LOAD. It asks whether the link is still live, then says
"opening this destroys it" and waits for a click. A page that revealed
automatically would be burned by the first scanner that fetched it - which is
the same reason redeeming is a POST.
A MISSING FRAGMENT GETS ITS OWN MESSAGE. If a link rewriter strips the part
after the #, the recipient is told the link is missing its key and what to ask
for, rather than being shown a generic failure. That is the one failure mode
this design cannot recover from, so it is worth naming to the person looking at
it.
The client-side guards live in securesend.js and are advisory by nature - the
server cannot see plaintext, so its size limit is what enforces. They refuse a
pasted FILE outright, which removes the easiest wrong path without anyone
reading a rule, and they warn on something document-shaped: over the length a
credential needs, more than twenty lines, or starting with a PDF/zip/image
signature.
Web Crypto only, no dependency, AES-GCM so a tampered ciphertext fails to
decrypt rather than producing plausible rubbish. The key is base64url for the
fragment, which survives copy-paste and link rewriting better than the standard
alphabet.
VERIFIED THAT THE PAGES RENDER, headlessly, against the dev server: the card
appears under Secrets & Access and the create form draws with its counter and
options. NOT VERIFIED END TO END - no secret has been created, sent, opened or
decrypted, because that needs the dev API restarted and the table migrated.
9845867 gave it its own plugin because Tech Tools is browser-only and this needs
storage. That was the right reading of the code and the wrong answer for the
person using it: Tech Tools is where you go for a utility, and a tool nobody can
find is a tool nobody uses.
So Tech Tools now has a blueprint, a table and a migration chain where it had
none. Its docstring and manifest say why, because "every tool here runs in the
browser" was load-bearing documentation and is no longer true.
WHAT MOVED
/api/securesend -> /api/tools/secrets
securesend.create -> tools.secrets
securesend0001baseline -> tools0001securesend
PLUGIN_TABLE_OWNERS, EXPECTED_HEAD_REVISION, EXEMPT_ENDPOINTS, the universal
profile and the api-inventory surface all follow it.
THE OLD "NO BACKEND" TEST IS NARROWED, NOT DELETED. It asserted
get_blueprint() is None, which is the sort of guard that gets removed when it
becomes inconvenient. It now asserts the blueprint serves EXACTLY the three
secret routes, with their methods - so a future tool that quietly grows an
endpoint or a table has to edit this test, and therefore make the decision on
purpose rather than by drift.
Asserting methods rather than paths also pins the detail this design turns on:
redeeming is POST-ONLY. Chat and mail clients prefetch GET urls to build
previews, and a preview that could redeem would consume a one-view secret before
its recipient opened the link. Previously that was a comment; now it fails a
test.
UNINSTALLING THE OLD PLUGIN COULD NOT BE DONE WITH THE PLUGIN TOOLING. `flask
plugin uninstall securesend` loads a plugin before uninstalling it, so a plugin
whose directory has been deleted cannot be removed by the command that manages
plugins - the registry entry had to be edited by hand. Worth fixing separately;
it is the same registry-versus-code divergence the device work hit from the
other side.
MIGRATING A CHAIN THAT HAS ALREADY RUN needs the old table and its version row
dropped first, because the relocated chain has no stamp and tries to create a
table that exists. Only relevant to anyone who installed securesend during the
hour it existed - it did reach the publication repo in 4242028 - and the fix is
DROP TABLE securesendsecrets plus DROP TABLE alembic_version_securesend before
running plugin upgrade-all.
1829 tests green. Still backend only: there are no pages yet, so nothing here is
reachable by a person.
The habit this replaces is pasting a credential into Teams, where it stays until
someone deletes the conversation. Tabled on 2026-08-21 pending a decision about
scope; the decision was text only, so this is the half that survives it.
BACKEND ONLY. There is no page yet, so nothing here is reachable by a person.
TEXT, NOT DOCUMENTS, and that shapes everything. Dropping files removed the
export-control question entirely - a password is not technical data - along with
blob storage, size and MIME handling, and the need to authenticate the recipient
so a document fetch could be pinned to a name.
TWO DOORS. Creating needs a login AND securesend.create: anonymous creation
would be an untraceable way to move data out of a system that otherwise records
who did what, and the creator is the only person these rows can ever name.
Reading needs neither, because the recipient usually has no account here and the
link is handed over on a call - the unguessable token IS the authorisation,
which is why it is 128 bits of randomness and why the row is spent on first read.
REDEEMING IS A POST, and that is the detail that would have broken this in
practice. Teams, Outlook and Safe Links prefetch GET urls to build previews and
scan for malware, so a GET that redeemed would let a preview window consume a
secret before its recipient ever saw it. The GET route that remains reports only
whether a link is still live, so the read page can say "opening this will
destroy it" without spending the only view to find out.
THE SIZE LIMIT IS THE WHOLE ENFORCEMENT. The key lives in the URL fragment,
which browsers never transmit, so the server holds ciphertext it cannot read and
cannot tell a password from a page of a drawing. MAX_CIPHERTEXT is therefore not
a tuning knob: raising it quietly turns this into the document transfer it was
scoped not to be. The refusal says what the limit is FOR, because somebody
hitting it is usually trying to send a file and the answer is a different tool.
Line counts, file signatures and refusing a pasted file belong in the browser,
guide an honest person, and are bypassable by anyone willing to POST directly.
NOT ENUMERABLE. Missing, expired and spent all answer identically - telling them
apart would confirm to anyone holding a guessed token whether it ever named
anything. A spent or expired row is deleted on the way past rather than waiting
for a sweeper, because it has no reason to keep its ciphertext.
Its own plugin rather than a Tech Tool: that plugin is browser-only by design,
and this needs storage, expiry and a view counter. Not an asset extension
either, so no assetid and no asset type - the scaffold assumes one and it was
removed.
FOUND BY ITS OWN TESTS: expiryhours=0 returned 201. `int(x or DEFAULT)` treats
an explicit zero as absent, so "expire immediately" silently became "expire in a
day". A caller who sends a number is now told when it is not allowed rather than
quietly given a different one.
BEFORE ANYONE RELIES ON THIS, test that your link rewriter preserves URL
fragments: create a secret, send the link to yourself in Teams, click it from the
message, and check the key is still after the #. If Safe Links strips it the
secret is unrecoverable and the server cannot help, which is the point of the
design. The fallback is passphrase mode, worth building only if that test says so.
Three product gates caught what a new plugin has to declare, and each was right
to: the route-parity gate wanted the endpoints described in api-inventory.json,
the lean-build guard wanted the plugin listed in the universal profile the
released installer builds from, and the authz sweep wanted the public redeem
route named in EXEMPT_ENDPOINTS with a reason. That last one matters most - a
mutating route that answers an unauthenticated caller is a decision somebody has
to write down, not a default, and reading a secret is a mutation because it
spends it.
Reported as a barcode rendering 176 by 0. A bar height of zero is what does it:
the number went straight into a CSS length, height: 0in is perfectly valid, and
an image sized height: 100% of a zero-height box is zero pixels tall. The label
comes out full width with nothing on it, which reads as a rendering failure
rather than as a field doing exactly what it was told.
A cleared field was the same bug wearing a different hat. An emptied number
input holds '', not 0, so the page emitted `NaNin`; the browser dropped the
declaration and the box fell back to auto height. That one merely looked wrong
instead of vanishing, which is arguably worse for finding it.
Measurements now go through labelVars(), which will not emit NaN, an empty
length, or a zero for anything whose zero means the box stops existing. Padding,
the quiet zone and the gap still take zero, because there zero is a real answer,
and offsets stay signed. It is a separate module because this is exactly the
kind of arithmetic that needs tests rather than a preview: every case in
labelVars.spec.js is a value a number input can actually hold.
The stylesheet now carries fallbacks on its var() reads as well, but they are a
net, not the fix: a custom property set to garbage is substituted and then
dropped at computed value time, and the fallback does not apply - it only covers
a property that is absent entirely.
Two separate floors were collapsing a live cartridge into a spent one, and
removing either alone changes nothing.
EMPTY_LEVEL was 5, so everything at or below 5% was assigned daysleft 0
outright. The reasoning was that a printer at 1% is out of toner as far as
anyone standing at it is concerned. But the row exists to say how long is
left, and 2% draining a point a day has two days in it. Flooring put that
cartridge beside ones that genuinely are empty with no way back to the
difference. It is 0 now: empty means empty.
int(level / rate) then truncated the division, so anything under a full day
arrived as 0 whatever the floor did - a cartridge with six hours in it was
indistinguishable from one with nothing, and the band read it as empty.
daysleft is fractional now, rounded to two places, which is about a quarter of
an hour: finer than the estimate deserves, but it costs nothing and keeps the
ordering of two nearly-spent cartridges meaningful.
daysText reads in whatever unit carries meaning: "6 hours", "1 hour", "29 min",
"4 days". Below an hour it goes to minutes with a floor of one, because
rounding hours would land back on "empty" - the same bug one rung down.
BEHAVIOUR CHANGE worth knowing: a cartridge at 1-5% with NO history used to get
daysleft 0 from the floor and land on the order list. A rate needs two
readings; with none there is nothing to divide, and the old answer was right by
accident - it said "empty" about a level nobody had watched move. It now
reports 'no history' and shows under "No estimate yet". This reaches only
printers newly added to Zabbix; anything that got to 3% the ordinary way has
the history to forecast from.
Five existing tests pinned the old rule. They recorded a real decision, so they
are rewritten to the new one rather than deleted. One of them was passing for
the wrong reason: its series (2.0, 1.7, 1.4, 1.1) drops 0.9, under
MIN_DROP_FOR_ESTIMATE, so it never had a rate at all and only passed because
the floor short-circuited ahead of the rate check. It now uses a real 20-day
drop at a tenth of a point a day and asserts the ten days its docstring always
described.
The forecast fixture's black cartridge moves from 1% to 0% so the empty band
keeps its API-level coverage, and a magenta at 3% covers hours-left end to end.
Reported as the CODE128 not appearing in the preview. It was appearing, in 44
of the 45 preset and layout combinations swept - as a 36 pixel stub. The code
box and the caption were both allowed to grow, so a caption of any length took
half the label and then kept taking, and on 1.00 x 0.50in stock a few words of
caption left a barcode too narrow to scan. Present, useless, and indisinguishable
from a failed render. The caption now takes the room it needs and no more, and a
CODE128 holds at least 40% of the label, or all of it when stacked: past that
there are too few narrow bars left to read, and a caption that wraps or clips is
the better trade.
So that a real failure can never again look like this one, an empty code box now
says so in red in the preview. It is screen-only - the marker must never reach a
printed label.
The settings were a narrow column with the preview trailing below them, which
put the labels being adjusted off the bottom of the screen. Settings on the
left, labels on the right in a column that stays in view and scrolls on its own,
folding back to one column on a narrow window.
That scrolling column is a print hazard worth naming: an ancestor with its own
overflow clips a multi-page print to whatever happened to be on screen. Print
resets it, and the check is a six-page two-sided run coming out as six pages
with the label as the only clipping box in the chain.
Reported as "CSF04 now reports all 0%", on the printer's own page as well as
the reports. Every supply on one host reading zero at the same instant is not
what a set of cartridges does; it is what a host that stopped answering looks
like.
getsuppliesbyip turned every unreadable item into 0:
try:
level = int(float(item.get("lastvalue", 0)))
except (ValueError, TypeError):
level = 0
An item that has never collected returns lastvalue as an EMPTY STRING, so
float('') raised and the handler substituted zero. lastclock was fetched in the
same request and read by nothing at all, so a reading from three weeks ago was
presented as current.
Zero is a legitimate level meaning spent, which is what made the substitution
dangerous rather than merely wrong. Downstream it reached `level <=
EMPTY_LEVEL`, which set daysleft to 0, which put the printer on the order list:
the report asked purchasing to buy a full set of cartridges for a printer
nobody had measured. On a waste container it was worse - waste is scored as
100 - level, so an unreadable one read as 100% full, the alarm state.
_readlevel now returns (level, lastseen) with level None for no usable reading:
empty, missing or unparseable lastvalue, or a lastclock older than 24h. A
lastclock that is absent is not treated as stale - a reading that cannot be
aged is not thereby wrong, and discarding it would lose real levels. A genuine
"0" is still 0, which is the half that keeps an actually-empty cartridge on the
order list.
classifysupply gains an 'unknown' status with remaining None. It is not
critical: critical is a claim someone acts on, and it may not be made about a
supply nobody measured.
The low-supplies report skips unknown rather than listing it as needing
replacement, and counts it in summary.unknown - a printer that has gone quiet
should be visible as that, not absent and not masquerading as empty.
_annotate_supply reads .get('level') with no 0 default, which would have put
the whole bug back.
PrinterDetail already rendered `level !== null ? ... : 'N/A'`, so it starts
telling the truth as soon as the backend sends null; the UI had been written
for a case the service never produced. It gains the unknown style - muted, not
a severity colour - and a tooltip separating "no reading at all" from "no
reading since <date>", which are different faults with different fixes.
Alerts needed no change: float(None) raises and that loop already continues,
so unknown supplies stop firing low-toner emails as a consequence.
12 of the 16 new tests fail on the old code. The 4 that pass either way pin
that real zeros still behave, which is the regression this fix could cause.
Not verified against CSF04's actual Zabbix data - dev has no reachable Zabbix.
This fixes the mechanism that turns unknown into 0%; /api/printers/<id>/supplies
now distinguishes them, with null for no reading and 0 for a measured zero.
The forecast identified a cartridge by its part number alone. 'W2020X' only
tells you which toner it is once you already know, and both places that name a
supply - the Order now list and the Part column - showed nothing else, so the
report read as a list of codes.
Both now show marketingname, which is what the box says: '414X Black'. The
part number moves to the title attribute, one hover away, because it is still
the thing an order is placed against. A supply with no name on file falls back
to the number rather than showing an empty cell; marketingname is nullable.
'Copy list' is deliberately untouched. It writes the part number verbatim,
which is what purchasing pastes into an order, and a spec pins that the name
does NOT reach that text.
firstPart() now returns the whole part row rather than its number, since the
cell needs both halves. It has one caller.
Dropped the .partnumber style with its .unmapped variant: nothing renders that
class any more, and the variant was already dead - the unmapped block lists no
part number by definition.
Verified by mounting the component against a fixture. The dev box has no
Zabbix, so the page itself has no rows to look at there.
Correction only helped on the global search page, and people search from the
list they are already on. Extending it turned out to be a shape question rather
than a volume one: thirty routes take a search parameter across a dozen files,
so carrying a suggestion in each of their responses is a large change today and
one more thing every future plugin author has to remember.
So the suggestion moved to ITS OWN ENDPOINT, /api/search/suggest, which any page
can call after rendering no rows. A page that has not adopted it shows nothing,
which is exactly what it showed before - nothing breaks by omission.
The plumbing lives in useListQuery, which already owned the search term, so a
list needs three lines: take `suggestion` and `reportCount` from the composable,
call reportCount(rows.length) after a fetch, and drop SearchSuggestion into the
empty state it already has. A list that never calls reportCount never offers a
suggestion.
Wired: global search, machines, printers, PCs, network devices, measuring tools,
knowledge base, vendors. NOT wired, deliberately: the type and reference lists
(machine types, PC types, VLANs, subnets, operating systems and the rest), which
are small controlled vocabularies nobody typo-searches, and USB, whose empty
state has a different shape and wants doing by hand rather than by pattern.
TRAP FOUND WHILE WIRING IT, and left commented in every page: applying a
suggestion by calling setSearch alone updates the box and the URL and does NOT
reload the list. setSearch only syncs the URL, and the watcher that would reload
is suppressed because search.value already holds the new term - the same trap
the global search page documents in performSearch. Each page calls its own load
function directly.
The composable guards a stale answer arriving after a newer search was typed,
never offers back the word that was typed, and swallows its own errors: a search
that found nothing is already the answer, and failing to improve on it is not
worth an error in front of anyone.
The route-parity gate caught the new endpoint being served without an entry in
docs/api-inventory.json, which is hand-written on purpose; added, and the spec
regenerated from it (283 paths, 418 operations). That regeneration also carries
the openapi version to 0.12.0, left over from the release.
The back side landed last commit carrying text by default, and a back laid out
as text-only renders no code box at all. So a two-sided run came out with a
blank reverse, which reads as the code having failed rather than as a setting
doing exactly what it said. The back now defaults to carrying a code, takes its
own content per row, and chooses its own symbology - a QR on the front that
opens the record, a CODE128 on the back that the bench scanner already reads.
Text-only is refused while the back carries a code, and the contradiction is
reconciled on load, so a saved setting from the broken combination repairs
itself rather than printing another blank box of cards.
A picture can go on a label: above the code, beside it, or behind everything as
a watermark, on either side or both. The file is read in the browser and never
sent anywhere, which is what makes it usable at an air-gapped site, and it is
deliberately not saved between visits - a data URL runs to megabytes and would
take every sizing value with it when the quota gave out. The watermark says
plainly that it costs the scanner contrast.
The generator was a full-screen route with no navigation and a narrow column of
controls against the left edge. That shape exists so a print cannot put the
sidebar on label stock, which is a real hazard and the reason core's label pages
are built that way - but it is a hazard the print stylesheet can answer, and
style.css now hides the shell at print time. A tool someone spends ten minutes
in gets the navigation and the full width; the stock still gets nothing but
labels, confirmed against a print PDF rather than a preview.
Two smaller things found while checking the above. A test print paired each page
with a rendered code by loop position, while the codes are keyed to position in
the whole run, so a two-sided test card could carry the wrong code; the index
now travels with the page. And the back's dotted preview outline outspecified
the print reset, drawing a dashed rule down the edge of every second card.
The corrector landed in fa0f6d7 unwired, pending a week of zero-result logs to
say what it should be aimed at. Attaching it now instead: the guards that make
it safe here are principled rather than tuned to any particular data, so the
logs would confirm the shape of the problem without changing the answer.
ONLY ON ZERO RESULTS. A search that finds something pays for none of this - no
vocabulary build, no scoring, no extra query.
The vocabulary is harvested from columns people actually search - asset numbers
and names, vendors, models, applications, asset types - cached ten minutes and
capped per column. Long enough that someone mistyping twice does not rebuild it,
short enough that a vendor entered this morning is correctable before lunch. The
cap is there because this exists to catch a typo against the names in use, not
to mirror the database into memory to answer one query.
SUGGESTED, NEVER APPLIED. The response carries the better spelling and the
caller decides. Searching for something else on a person's behalf is how
somebody orders the wrong cartridge.
FAILS SOFT. A missing column on a lean install, or anything else, returns no
suggestion and the search answers normally. No vocabulary is worse than the
alternative; it is not fatal.
Three tests through the endpoint rather than only the unit: a misspelling gets a
suggestion, a search that succeeds gets none, and CSF18 against an existing
CSF17 returns no results AND no suggestion - the digit guard holding across the
whole stack, which is the one that would cost somebody a wrong bay.
The zero-result logging stays. It answers "what can people not find", which the
corrector does not: a typo is only one of the reasons a search comes back empty,
and the others are vocabulary nobody has entered and records that do not exist.
NOT DONE: the search page does not render the suggestion yet, so this is
invisible to a person until that Vue change lands.
Two halves of one question - how to handle a misspelled search - deliberately
kept separate, because only one of them should be turned on today.
WHAT NOBODY CAN FIND IS NOT RECORDED ANYWHERE. A search returning zero results
is the only evidence of the gap between what people look for and what is there,
and it vanished. Logged now at INFO with a stable prefix, so a week of it greps
into a list. That list is what should decide whether correction is worth wiring
in: multi-word matching and cross-field matching both changed in the last day,
so a good share of what used to fail may already be found. The failures left
over might be typos, or vocabulary nobody has entered, or records that genuinely
do not exist - and each wants a different answer.
THE CORRECTOR IS BUILT AND NOT WIRED IN. shopdb/core/services/spellfix.py, with
tests, ready to attach in about ten lines once there is evidence about what to
attach it to.
NOT SOUNDEX, which was the obvious candidate. MySQL has it and SQLite does not,
and the suite runs on SQLite while production runs MySQL - so a SOUNDEX() in a
query is either an error in every test or a production path no test executes.
It is also wrong for this data: soundex is English-name phonetics, four
characters wide, and it DISCARDS DIGITS, so CSF16 and CSF17 hash identically.
Half of what people search here is an identifier.
So: character distance in Python, same behaviour on both dialects.
THE VOCABULARY IS THE DATA. No dictionary holds Genspect, Telesis, Keyence or
wax-trace. Terms come from the columns being searched, which also means a vendor
added this morning is correctable this morning.
TWO GUARDS, and they are the point rather than a detail. Digits must match
EXACTLY: CSF16 to CSF17 is one edit and a different bay, so anything carrying
digits is either right or not correctable - while cfs16 to csf16 still works,
because the guard is on the digits and not on identifiers wholesale. And the
first character must match, since typos land mid-word far more often than on the
first key, which costs almost no recall.
Distance is Damerau-Levenshtein so a transposition costs one edit rather than
two - Keyecne for Keyence is the commonest error there is, and plain Levenshtein
pushes it past the threshold on short words. Allowed distance scales with
length. A tie returns NOTHING: two equally good candidates means there is no
answer, and offering either implies a confidence that is not there.
It suggests; it never rewrites. Silently searching for something else is how
somebody orders the wrong cartridge.
13 tests, weighted toward the refusals, because those are the cases where being
wrong costs something.
2bf1531 fixed the reported symptom and introduced the opposite one: markers that
had been landing left came right, and then sat well ABOVE where they belonged.
It was the right diagnosis and the wrong remedy. The drawing is painted with
object-fit: contain inside a fixed 390x300 preview, so it is letterboxed on one
axis and a percentage of the BOX is not a percentage of the DRAWING. That part
was true. But the fix resized the frame to shrink-wrap the image, and the
centring transform above it translates by percentages - which resolve against
that same element. Making the frame image-sized silently broke the transform's
assumption that the frame is the box. Width still matched at 390, height no
longer did, which is exactly x-right and y-wrong.
The frame fills the box again and object-fit: contain is back. Where the bars
fall is now computed instead: paintedBox works out the fraction of the box the
drawing covers and the gap before it, per axis, and the marker maths folds that
in. The transform's assumption stays true because the element it translates
never changes size.
Checked per axis rather than asserted. A tall drawing pillarboxes, so x
compresses and offsets inward - a left edge at 2 percent of the drawing lands at
13.1 percent of the box, which is where the drawing actually starts - and y is
untouched. A wide drawing does the reverse. A level shaped like the box is a
no-op, so nothing that already looked right moves. Centre maps to centre on
every shape.
NEW COUPLING, deliberately noted at both ends: PREVIEW_WIDTH and PREVIEW_HEIGHT
must stay in step with .map-preview in the styles. Resize the preview without
touching them and this drifts back.
Still not confirmed against a live instance, and this is the second attempt at
this bug, so it wants eyes on three cases: one that was landing left, one that
was landing high, and one that was right all along.
Reported from the floor: some assets showed their hover mini-map marker way off
to the left, outside the map, while the SAME asset sat correctly on the main
map. The data was right; the preview was drawing it wrong.
The preview is a fixed 390x300 box and the drawing was painted into it with
object-fit: contain, which letterboxes or pillarboxes to keep the drawing's own
aspect ratio. The marker, though, was positioned at a percentage of the BOX. So
the marker and the drawing were measured in two different coordinate spaces, and
they only agreed when the level's aspect ratio happened to be 390:300.
A level near that ratio looked perfect - 3300x2550 is 1.294 against the box's
1.300 - which is why this read as "some devices" rather than "the feature is
broken". A taller drawing pillarboxes, every marker's x shifts, and one near the
left edge lands in the grey gutter beside the map.
The image now SIZES the frame instead of filling it: the transform box
shrink-wraps the image and the image scales down to fit. A percentage of the
frame is therefore a percentage of the drawing by construction, at any aspect
ratio, and the letterboxing moves out to the preview around it where it does no
harm. dimensionsFor still converts the stored pixels to percentages - that part
was always right.
NOT visually confirmed against a live instance. The reasoning is traced and the
build is clean, but the shape of this bug is that it looks correct on any level
close to the old box ratio, so it wants a look at an asset that was landing left
and one that was already fine.
Related and NOT addressed: the wheel zoom applies translate percentages that
resolve against the element and are then multiplied by scale, so a zoomed
preview may still drift. At rest, which is the reported case, it is exact.
An article's searchable text lives in three places: its title, its keywords,
and the name of its topic. Nobody typing a search knows or cares which word
came from which, so 'CMM Community' means "the article tagged community, under
the CMM topic". Both search paths returned nothing for it.
The plugin's own listing, which is what the KB page calls, built a single
ilike('%CMM Community%'). That needs the whole phrase contiguous in ONE field,
and no article has it: 'CMM' is only in the topic's name, 'Community' only in
the keywords. Global search already split the query into words but only ever
looked at title and keywords, so a word that only the topic could satisfy
failed there too - the same bug wearing a different face.
Every word must now be found somewhere across all three fields, in any order.
_word_match gains an `extra` hook for a word that a RELATED row satisfies
rather than a column of this table, which is how the topic joins in without
colliding with the sort=='topic' join.
The retired-topic rule is unchanged and pinned on both paths: matching more
words is not a way past it.
Verified against the 235-article dev library, where all three of these
returned nothing before: 'Fieldglass Jefferson' (title word plus topic word),
'outage notification' (two keywords, not adjacent), 'compucom Jefferson'.
The same phrase-only pattern is repeated in about ten other list endpoints
(computers, machines, printers, network, printedparts, measuringtools, usb,
notifications). Left alone here - fixing them properly means promoting the
word-match helper onto the shopdb.api contract surface rather than copying it
per plugin, which is a contract bump.
Evaluated a Password-Pusher-style service as a Tech Tool and tabled it. The note
carries the decisions rather than just the idea, because those are what cost the
time: it is NOT a Tools-plugin candidate as it stands (that plugin is browser
only by design, with get_blueprint and get_models both empty), the sender must
always be authenticated, a FILE recipient should be logged in too - which costs
nothing on an intranet-only server where anyone who can resolve the link already
has an account - and zero-knowledge encryption is right for a credential but
questionable for a controlled document, where a server that cannot say what was
shared is a liability rather than a feature.
Intranet-only hosting is what makes the idea viable at all: a link mailed
outside is dead and no ciphertext reaches a third party. Pointing staff at the
public hosted instance instead would reinstate every problem this avoids.
Three things the code generator could not do, each of which had someone
working around it with a control that meant something else.
The gap between a code and its caption was the quiet zone. That is the
clear space a scanner needs around the code, so closing up a caption meant
shaving the thing that makes the code readable, and the only way to get a
tight label to scan was to push the caption further away than it looked
like it should be. Quiet zone now means the code's own margin and nothing
else; gap means the distance to the caption, and adds to the quiet zone
rather than eating into it. Each stock preset carries the gap that suits it.
Position had one pair of controls and they moved the label box on the
stock. That is the fix for a printer whose origin is off, and it is not the
fix for a code sitting in the wrong place on a label - but it was the only
lever there, so it got used for both. The label div is now the media and
holds still; everything printed sits in a content wrapper that Align,
Vertical and Content X/Y move within it. The old pair kept its behaviour
and is labelled Media X/Y.
A badge card has two sides and the generator only ever emitted one page per
card. The back can now carry the same code again, a fixed block of text,
its own per-row content, or nothing. Page order is a choice because the two
kinds of printer want opposite things: interleaved for a duplex card
printer, which takes pages two at a time, and grouped for a single-sided
one whose stack gets flipped and reloaded. A row that leaves its back
column empty still gets a blank back page - a page count that varies per
row shifts every later back onto the wrong front, which prints a box of
perfectly readable cards that are all wrong.
CSV parsing and page building moved to plugins/tools/frontend/labelPages.js
(committed in 875fde9), where the column contract and both page orders are
covered by tests rather than by looking at the preview.
A review of d60ed60 and 8b9b936 found five things. Three were introduced by
those commits.
REFUSING A NAMED DEVICE MADE A MARKER PC CLAIM ITS OPERATION. _sync_partmarker
returned [] both for "not a marker PC" and for "a marker PC that linked
nothing", and the caller reads [] as the first - so a bay whose asset-id.txt
named something unresolvable fell through to the ordinary machine link, and for
a marker PC the machine number IS the operation. It took an active link to a
record that can only have one holder while several markers share it, and the
warning said "not linked". The previous behaviour minted a twin; this traded
that for a contested operation. None now means "not a marker PC" and is the only
answer that lets the machine link run; the response normalises it away so the
API shape is unchanged.
A DORMANT CHALLENGER WAS PROMOTED BY DELETING A FILE. Recording a challenger
dormant leaves a row that the next cycle finds as `reuse` and reactivated with
no incumbent check - so a second PC took a live device by its enrollment file
becoming unreadable. Both reuse branches re-check incumbency now, which is what
_sync_machine_link always did.
AN INCUMBENT UNDER ANOTHER COLLECTOR LABEL WAS INVISIBLE. Incumbency was queried
on our own label, but on a CMM the instrument IS the reported bay, so the
incumbent's link is the machine sync's row. A second PC naming that instrument
found no incumbent and linked actively: two live holders of one instrument, each
invisible to the other. Incumbency now counts any collector-owned label. A row
made BY HAND carries none of them and is still excluded - a person's link is not
the collector's to archive.
THE NETWORK FORM LOCKED OUT THE ROWS IT NEEDED TO FIX. Asset number is disabled
while editing, correctly, but the payload is built in script so the blank was
still sent - and the new server-side guard rejects it. A device with no asset
number could not be saved at all, and the field could not be typed into. It now
unlocks only for a record that loaded without one, with a hint saying why.
"2 in 4.0d" WAS THE LABEL LYING. Replacements are counted across the whole
history window; basisdays is only how long the current cartridge has been in.
Joining them with "in" claimed two changes inside four days - the exact shape
reported as unbelievable, except here the data was right. Now "2, this one 4.0d".
Two toner dips the same review found:
A MULTI-POLL OUTAGE STILL MINTED A PHANTOM SWAP. Only single readings were
dropped, so 90, 0, 0, 90 survived and 0 -> 90 scored as a change. Dips of any
length are handled now. One bad sample stays a candidate whatever the polling
cadence, because a reading is an instant; several consecutive low ones only
count as one outage when they are close together, since days at zero is a real
empty period. That time bound also separates an outage from a swap, ordinary
consumption, and a second swap, which have the same shape in levels alone.
THE DIP FILTER ATE REAL SWAPS OF NEARLY-FULL CARTRIDGES. Recovery was tested
with an absolute difference, so 95 then 5 then 100 read as a recovery because
100 and 95 are close, and the swap evidence was deleted. Toner only falls: a
recovery comes back at or BELOW where it left, a new cartridge comes back
higher.
The review also proved by reverting each feature that the previous tests did not
pin the median burn rate or the near-full rule - both passed with the bug
restored. Verified by the same method that all four toner behaviours now fail
when reverted, and the burst assertion is tight enough to tell 0.2 from 0.88.
Reported from the floor: two printers showing 81 and 83 percent, both forecast
to run out in a fortnight. That is a rate near 5.8 percent a day on a cartridge
barely touched.
burn_rate required four readings and a 2 point drop, and checked only that some
time had passed - not how much. Supply items are commonly polled every few
minutes, so four readings can span a quarter of an hour, and a 2 point drop
across fifteen minutes extrapolates to nearly 200 percent a day. The report then
sorted that confident wrong number into "soon", next to cartridges that really
are about to run out. It did not look broken; it looked urgent.
A rate now needs two days behind it. Two days is the smallest span that survives
a printer's daily rhythm, so one heavy morning does not become the whole picture.
Below that the answer is "not enough history yet", which the report already has
a home for: band() returns None and the row lands in the "No estimate yet"
section with its reason shown, rather than competing for attention with real
urgency.
This is the third face of the same fault. A phantom replacement truncated the
run - "2 replacements in 2.3 days" was the same printer saying so - and the rate
was then fitted to whatever short stub remained. The near-full rule and the dip
filter stop the truncation; this stops a stub from producing a number at all.
Two things reported from the floor, one cause each.
"5 CHANGES IN 90 DAYS, THAT'S HARD TO BELIEVE." It was. A replacement was any
+10 rise between readings, with no check on where it landed, so two shapes that
are not swaps scored as swaps: a supply reading 0 or near-0 while it was out of
the machine and then reading normally again, and a coarse gauge ticking back up
after a reseat or a power cycle.
A swap must now also LAND near full, because that is what a new cartridge reads,
and a single dip that RECOVERS to roughly where it came from is dropped before
anything looks at it. The dip filter keys on shape rather than cause, which is
why it holds for all of them - a supply pulled out to be shaken, a door open
mid-poll, or a site whose preprocessing maps the Printer MIB's unknown
sentinels onto 0. It is NOT a Zabbix timeout: an item that does not answer
records nothing rather than writing a zero. A genuine near-empty reading before
a real swap does not recover, it jumps to full, so it survives and its swap
still counts.
find_replacements and current_run now read one predicate. When they disagreed, a
phantom rise reset the run and threw away the history the estimate needed - so
the bad count was quietly damaging the rate as well, which is why both were
wrong at once. Expect replacement counts to FALL and per-cartridge history to
lengthen.
A BURST BIASED THE RATE FOR THE LIFE OF THE CARTRIDGE. The rate was the slope
between the first and last reading of the run, and two endpoints cannot tell
"steady" from "burst then stopped". A cartridge that lost 20 percent in two days
and then barely moved for a month read as 0.83 percent/day forever after, so the
report kept promising it would run out long after printing slowed. It is now the
median of the per-interval rates: the burst is one interval among many rather
than one of two points. Rising intervals are dropped as noise; flat ones stay in
at zero, because a cartridge that did not move is real information. If every
interval is flat or rising yet the run dropped overall, it falls back to the
whole-run slope rather than reporting nothing.
Where the intervals disagree by 5x or more the rate carries a marker and an
explanation on hover. The number is still the best estimate available; the flag
stops it reading as a measurement.
The "Changed" column is "Replacements", and its cell says "2 in 90d" rather than
"2 / 90d", which was read as a date, a ratio and a version number.
A review of last week's device-identity work found these; two were writing bad
data and one was reproduced against a live server before being fixed.
AN UPDATE COULD BLANK AN ASSET NUMBER, on all six asset update paths. Create
validates it and the column is NOT NULL, but the conflict check only runs when
the value DIFFERS, and '' collides with nothing - so an empty assetnumber went
straight through to a required column. This is the likely source of the assets
found with no number: a form that loaded blank and was then saved.
MACHINEFORM COULD LOAD BLANK AND LET YOU SAVE IT. One try/catch wrapped eight
reference loads AND the machine fetch, so a single transient failure among them
- one page of listAll() timing out during a collector cycle is enough - rejected
the whole block and rendered a fully editable EDIT form with every field empty,
the error banner far below next to Save. Typing an asset number and saving then
wrote the blanks over a real machine. The record now loads in its own try, and a
failure shows the reason INSTEAD of the form: an empty edit form is
indistinguishable from a record whose fields are genuinely empty.
NAMING A DEVICE THAT DID NOT RESOLVE STILL MINTED A TWIN. Both device paths
warned "not linked" and then fell through to mint <HOST>-PARTMARKER or
<HOST>-CMM - the hostname-derived twin the resolution order exists to prevent.
The warning was true about the typo'd number and false about the twin. Naming a
device is a commitment: if the name does not resolve, or resolves to the wrong
kind of thing, link nothing and say so. Silence still means "work it out", so a
bay with no file keeps the reuse-then-mint behaviour it always had.
TWO PCS COULD BOTH HOLD ONE DEVICE, ACTIVELY, WITH NO WARNING. Verified against
a live server: report as one host, then as another naming the same marker, and
both controls rows stayed active. Neither device path had ever looked at who
else held the target - only at links whose source was THIS PC - so a replaced PC
kept its link forever and an asset-id.txt copied to a second bay claimed the
device silently. It now reuses the machine link's rule rather than inventing a
second one: an incumbent that has gone quiet past the claim window or been moved
off In Use has yielded and is archived, never deleted; a live incumbent keeps
the device and the challenger is recorded dormant.
The swap test asserted the old behaviour and now asserts the new one, split in
two: a live incumbent keeps it, and handover completes once the incumbent
yields. Two other tests were passing while their names lied - the unknown-device
one checked only that the typo'd asset was not created, not that nothing was
linked, and it passed while a twin was minted beside it.
Three fixes, all of them a page not doing what its siblings already do.
MODELS WERE NOT FILTERED BY VENDOR. PCForm, MachineForm and PrinterForm each
narrow the model list once a vendor is chosen; NetworkDeviceForm bound the whole
catalogue, so picking Palo Alto still offered every Dell and Zebra model. Same
computed as the others, including the same rule that no vendor selected shows
everything - an empty dropdown reads as "no models exist" when it means "pick a
vendor first". Audited the rest: this was the only gap. The other views holding
a modelnumberid have no vendor picker to filter against, and settings ModelsList
is where a model's vendor is ASSIGNED, where filtering would be circular.
THE HERO RAN THE LABEL INTO THE VALUE - "Asset #FW-OAV..." as one string. The
page used detail-item / label / value, which match nothing in the stylesheet, so
the two spans got no layout at all. Every other detail page uses hero-detail /
hero-detail-label / hero-detail-value, which stacks a small uppercase label above
the value. Renamed to those; no CSS added, because the styles already existed and
this page simply was not using them. It was the last page using the unstyled
names.
MACHINES LIST LINKED BY THE WRONG ID on its fallback path. `/machines/:id` keys
on machineid, the plugin extension id, and the row click and View button fell
back to `item.assetid` - which lands on whichever machine happens to carry that
number: a wrong page that looks right, which is worse than a 404. That defect has
been fixed twice before in other views (AssetRelationships, then the GE-Enforce
reports table) and BackupHistory carries a comment warning about it; this was the
fourth copy. The list endpoint always sets item.machine, so the fallback could
not actually fire here - it is removed as a latent trap rather than a live bug,
and with no machineid the cell now shows plain text rather than a link that
misleads.
Phase 3 was written as "the collector reads a registry value". It is all three
components moving together: imaging writes DeviceId, GE-Enforce resolves device
identity registry-first, and the collector reads the registry ahead of the file.
The argument for it is already in the fleet's own code. Install-FromManifest
resolves TargetMachineNumbers gating from the eDNC registry BEFORE
machine-number.txt, and says why: the file is written ONCE by startnet.cmd at
the PXE menu and is not updated on reassignment, so it goes stale.
asset-id.txt inherits that defect exactly. Nothing rewrites it when a device is
swapped, moved between bays or replaced, and phase 1 put it at the TOP of the
resolution order - so a stale value is adopted with no warning, because it still
resolves. A file only imaging writes cannot be the top of a resolution order for
something that changes during a PC's life.
Also records the consequence: phase 3 needs a reassignment path the way
Set-MachineNumber exists for the bay number, or it moves the staleness from a
file into a registry value instead of curing it.
The phase 3 section named the key and left the scope to be inferred from the
fact that deviceid is one field. Spelling it out: DeviceId is a single value,
not a key per device type, and the pc-type decides which sync consumes it - so
CMM, Keyence, Genspect, wax-trace and part markers all read the same value, and
a family declared later through subordinatedevice_<pctype> is covered the day
it is declared.
Also records what the value REPLACES per family, which differs: for a CMM it
replaces cmmid.txt as identity only, for a part marker it replaces minting from
the hostname, and for the other three it replaces nothing because they never
had an identity at all.
Phase 1 shipped without a plan around it, and the next two steps both touch a
different repository and the live boot image, so the reasoning needs to outlive
this week rather than being rediscovered from the code.
The finding that shapes the whole thing: cmmid.txt CANNOT simply be retired. It
is a PATH KEY, not just an identity - Verify-And-Heal-Staging stages
installers-post\cmm\backups\<cmmid>, 09-Setup-CMM locates the staged set by it,
and Restore-CMM restores from <BackupRoot>\<cmmid> as a same-bay operation. Only
one of its five consumers is an identity use. That one is also the one doing the
job badly, reporting a CMM's INSTRUMENT id in the machinenumber channel so the
server can resolve it back to a measuring tool at step 4 of its own fallback
chain.
So phase 2 has imaging write asset-id.txt ALONGSIDE cmmid.txt - two files, two
questions - and demotes the identity use while every path use stays exactly
where it is. cmmid.txt survives phase 3 too; it is not on any deletion list.
Phase 3 is the registry, HKLM:\SOFTWARE\GE\ShopDB\DeviceId. No new key and no
new read path: the collector already opens that key for BaseUrl and ApiToken and
Install-GEEnforce already writes it. The proposal argues for doing phase 3
BEFORE phase 2 rather than after, because then imaging writes the registry for
new bays and skips the file era entirely - the file survives only for bays
staged between phases, and the cleanup is deleting a branch instead of sweeping
a fleet.
Also recorded: the rail that must not be crossed (machinenumber is what
TargetMachineNumbers gates on, so a device id in that channel stops every
bay-gated entry from matching, silently, forever), what each phase does NOT
change, per-phase rollback, and a note that phase 1 stops new duplicates without
merging the ones a PC swap has already made - so the reconciliation backlog
should be read BEFORE imaging starts writing identities over the top of it.
A bay with an NTLARS/eDNC MachineNo registry is identified by it, and a CMM by
cmmid.txt. Everything else - Keyence, Genspect, wax-trace, part markers, and
whatever a site declares next - had no stable identity at all, and the two
mechanisms that stood in for one both key off the PC: reuse looks for a prior
link from THIS PC asset, and minting builds `<PC number>-<SUFFIX>` where a PC's
number is its hostname.
Both survive a re-image. NEITHER survives a PC SWAP. A new hostname is a new PC
asset with no prior link and a predicted number that has never existed, so the
same physical device gets a second record while the first keeps its config and
its backup history under a dead PC's name. That is how 43 legacy MT-#### tools
ended up shadowed by minted twins, three records deep in places. The metrology
path learned this and gained an explicit id file; the part-marker path was
modelled on the metrology path as it stood BEFORE that fix, so it inherited the
defect - and its own docstring said so, describing minting "the same way it
already mints a CMM".
C:\Enrollment\asset-id.txt is now that identity for both, and for anything
declared later through subordinatedevice_<pctype>. It holds one line, the
device's assetnumber, and deliberately does NOT record what kind of device it
is: the pc-type already does, so a new device type needs no new file and no
client change. Resolution puts it first, ahead of everything PC-derived.
NOT machine-number.txt. machinenumber answers "which bay is this" and is what
GE-Enforce TargetMachineNumbers gates on, so naming a device there would
silently stop every bay-gated manifest entry from matching. A part marker still
files partof the operation from machinenumber; asset-id.txt changes which marker
the PC controls, not which operation that marker belongs to.
The wire field is `deviceid` rather than `assetid`, because assetid is already
this contract's RESPONSE field for a PC's integer primary key - the two would
have sat side by side meaning different things. measuringtoolid, shipped in
0.12.0, is accepted as an alias and the client still reads the older file, so a
bay staged in the last day keeps reporting; deviceid wins when both arrive.
Two guards, shared by both device families: a value that resolves to nothing
warns instead of minting a phantom, and a value that resolves to the WRONG KIND
of asset is refused with the asset named, so a machine number pasted into the
file cannot be filed under a device label the collector also owns.
Row creation is now one helper instead of a copy per path, because the named and
minted branches both need get-or-create on the extension row and the control
link.
1744 tests green, including a new file that pins the swap case both ways - with
the file one device, without it two. Five of its tests failed first time because
the fixtures built bare assets with no extension row and the type guard refused
them, which is the guard working.
VERIFIED ON WINDOWS 11 (build 26200), five cases: the new file alone; both files
present, new winning; the legacy file alone; neither, sending no field; and a
padded value with a trailing line.
The server has accepted `measuringtoolid` since the adoption work landed, and it
is the FIRST entry in the resolution order precisely because it is the identity
that survives a PC swap. Nothing ever sent it. The reporter read the eDNC
registry, cmmid.txt, machine-number.txt and pc-type.txt, and its own comment
said metrology bays have no per-bay id and therefore send nothing - so a Keyence
or Genspect bay fell through all four steps to minting, which the server's own
docstring calls the last resort.
Minting derives the asset number from the HOSTNAME, so a permanent instrument
inherits the identity of whichever PC drove it that week: replace the PC and
either the number lies or a second tool appears for the same physical unit. That
is how 43 legacy MT-#### tools ended up shadowed by minted twins. The half that
prevents it was built, tested and undeliverable.
The reporter now reads C:\Enrollment\measuringtool-id.txt and sends it when
present. Its own file, NOT machine-number.txt: machinenumber answers "which bay
is this" and is what GE-Enforce TargetMachineNumbers gates on, so naming a tool
there would silently stop every bay-gated manifest entry from matching.
The paste-ready reporter in COLLECTOR-INTEGRATION.md is a second implementation
of the same payload, so it gets the same resolver rather than being left to
drift. The field was also missing from the payload table and from the classic
api.asp mapping, and there was no prose anywhere describing how a tool is
resolved - added, including why minting is last and what the two guards refuse.
VERIFIED ON WINDOWS 11 (build 26200), four cases: a named instrument is read and
sent; no file sends no field and exits 0; a whitespace-only file behaves as
absent rather than sending an empty string; and a padded value with a second
line yields the first line trimmed.
Printers become a property of the bay, and the container stops throwing away the
site's own files every time it is updated.
Printer assignment is now end to end. The assignment belongs to the MACHINE and
reaches whichever PC controls it, so a reimaged or swapped bay comes back with
its printers and nothing had to be saved off the old box. One picker serves both
forms rather than two implementations of the same override. A relationship type
can declare itself singular, so setting a second default REPLACES the first
instead of silently losing to the older row. Alongside the assigned half there is
now an observed half: a bay reports what it actually has, kept strictly apart
from what it is told to have, because a drifted bay's own state becoming its
desired state would make every configuration error permanent. The client script
corrects a drifted queue in place rather than only installing a missing one, and
a driver rollout can be spread across waves so 300 bays do not pull 30 GB through
one five-minute window.
DOCKER SITES SHOULD READ THIS PARAGRAPH. The compose stack never persisted the
instance directory, so `docker compose build api && up -d api` - the update path
the docs themselves gave - discarded plugins.json and every upload with it. It is
a volume now, and DEPLOY.md carries the one-time rescue for a stack that predates
it. The image also could not be built at all: the frontend stage never copied the
plugin staging script its own prebuild hook runs, so every build since that
script landed failed. Air-gapped sites need a fresh offline bundle, because the
stack moves to MySQL 8.4 LTS and an existing tarball carries only the 8.0 image.
The Windows installer already bundled MySQL 8.4, so this closes a gap between the
two halves rather than moving anyone.
Adds a topology migration guide: IIS to Docker, Docker to a new host, and back.
The data moves cleanly; what costs time is that the server address is baked into
GE-Enforce manifests, the generated collector script, the printer client scripts
and printed QR codes.
1736 tests green. Contract stays at 0.20.0 - nothing under shopdb/api changed.
INSTALL-WINDOWS-IIS.md has said MySQL 8.4 LTS is standard for new installs since
8.0 reached end of life in April 2026, while both compose files and the offline
bundler still pinned 8.0. A site reading the Windows runbook and a site reading
the Docker one were being told to run different servers, and the migration page
written this week sent people onto the dead one.
Verified against a real server rather than by editing a tag: 8.4.11, core chain
plus five plugin chains applied clean, 66 tables at a single utf8mb4_unicode_ci
collation, six alembic version tables. The image's PyMySQL authenticates against
8.4's caching_sha2_password, which is what requirements.in already pins
cryptography for.
Existing servers need one thing done FIRST: 8.4 removes mysql_native_password,
so an account created on 5.6 or 5.7 must be moved to caching_sha2_password
before the upgrade or it cannot authenticate afterwards. In-place also has no
downgrade path, and 5.7 cannot reach 8.4 in one hop. For databases this size a
dump into a fresh 8.4 server is the better trade: same outage, and the old
server stays as the rollback.
Air-gapped sites need a fresh offline bundle, because the tarball carries the
MySQL image alongside the app image.
Also here, found by having it bite during that verification: the db healthcheck
pinged over the unix socket, and the entrypoint's init pass answers on the
socket while running the server with --skip-networking. The probe therefore
reported healthy DURING init, which is what `depends_on: service_healthy` gates
api and migrate on. A ping passed at 8 seconds and the next query failed because
the server was mid-restart. Probing 127.0.0.1 keeps it red until the real server
is listening.
The pieces existed across three pages and nothing connected them, so "can we go
from IIS to Docker" had no answer to point at. It is a fair question with a
short answer: the application keeps state in exactly two places, the database
and the instance directory, and nothing is encrypted at rest with SECRET_KEY or
JWT_SECRET_KEY, so a move is a dump plus a directory copy. The schema is
identical across topologies.
What the page spends its length on is the part that is NOT the data, because
that is where the time goes. The server address is baked into things that are
not the server: GE-Enforce manifests, the generated collector script, the
printer client scripts, and printed QR codes, which cannot be swept at all.
Keeping the hostname and repointing DNS makes the migration invisible to the
fleet; changing it does not.
Three other traps, each of which has a symptom that shows up later rather than
at cutover: an aliased IIS site needs MOUNT_PATH and a dist built for that
subpath, while the image builds for the root; an older dump can carry latin1 or
3-byte utf8 table definitions that load quietly into a utf8mb4 server and only
misbehave on the first accented name; and a restored instance directory needs
chown, because docker cp writes under the copying user's uid and the container
runs as shopdb.
Covers both directions plus Docker to a new Docker host, and ends with a cutover
checklist that leaves the old stack stopped rather than removed until a bay has
checked in on a working day.
Also fixes two links in START-HERE that pointed at files which are not there:
the ADR index needed its adr/ prefix, and LLM-GUIDE.md is llms.txt.
`db_data` was a volume and the instance directory was not, so the documented
update path - `docker compose build api && up -d api` - recreated the container
and discarded everything the site had written. `plugins.json` is only the loud
part: maps, branding, model and application images, employee photos, warranty
proofs, slides, printed-part files and the Dell OAuth token all live under
instance_path too. MySQL rows survive and point at files that are gone, so the
second symptom is images 404ing rather than an error anybody sees.
Reported by an adopting site, which read it as having updated too fast. It had
not; nothing it could have done differently would have kept those files.
DEPLOY.md had been telling sites to back up `instance/` since it was written.
The template never gave them anything to back up.
The air-gap `migrate` service mounts the volume too, because
`flask plugin upgrade-all` rewrites plugins.json and that service exits
immediately after.
The image now creates instance/ ITSELF, owned by the app user. Docker seeds an
empty named volume from image content at the mountpoint, ownership included;
with no such directory in the image the mountpoint is created root-owned 0755
and the container, which runs as shopdb, cannot write into its own instance
directory. Caught by running the built image rather than by reading it: the
volume mounted clean and `touch` came back Permission denied. Verified fixed the
same way.
A stack that predates the volume needs its files moved across ONCE, while the
old container still exists - the volume is seeded from image content, and the
image ships instance/ empty, so it comes up empty rather than inheriting the old
container's writable layer. DEPLOY.md carries the procedure, including the chown
after `docker compose cp`, which writes files under the copying user's numeric
uid rather than the app user's.
Also here, found while checking what an upgrade actually runs: the connected
update steps ran `flask db upgrade` and stopped. Per-plugin Alembic chains
(ADR-008) are not part of that, so a connected site taking an image with a
bumped plugin migration ran the core chain and silently skipped every plugin
chain. The air-gap stack had it right all along. Both commands are in Step 9
now, plus a `db current` check against `db heads`.
`npm run build` fires a `prebuild` hook that runs
`node ../scripts/stage-frontend.mjs`, the script that copies each plugin's
frontend into the Vite tree and codegens routes.gen.js. The Dockerfile's
frontend stage copied `frontend/` alone and flattened it to the stage root, so
`../scripts` resolved to `/scripts`, which does not exist. Every image build
since the staging script landed has died there on `Cannot find module`.
The stage now keeps the repo-relative layout - frontend/, scripts/ and plugins/
as siblings under /build - because the script resolves its repo root as its own
directory's parent. The final stage copies from /build/frontend/dist to match.
Verified with a real build, not by inspection: 16 plugin frontends staged, vite
produced dist, and the finished image carries frontend/dist with 241 assets.
This also unblocks SITE_PLUGINS lean image builds, which until now only
scripts/build-site.sh could do.
GE-Enforce gives each PC a start offset of SHA256(hostname) % 5 MINUTES and then
repeats every five minutes. That was sized for reading a few KB of manifest
JSON. A driver set is 100 MB for the two universals and 226 MB for a full site,
so the day a driver entry lands, every bay pulls it inside one five-minute
window: roughly 30 GB across 300 bays, at something like 800 Mbps, on the same
share the whole floor needs for everything else. The failure mode is not slow
printers, it is a floor that stops converging.
-WaveStart with -Waves spreads that out. Each PC derives its own wave from its
hostname, so there is no central coordination, no per-bay configuration, and no
list of who has had it yet. The hash is the same idiom Register-GEEnforce.ps1
already uses for its start offset, SHA-256 rather than MD5 because FIPS-enforced
bays disable MD5 outright and would throw.
Measured over 300 hostnames at 10 waves: 23 to 44 bays per wave against a mean of
30, so the peak wave moves about 4.3 GB rather than the 3 GB an average implies.
Hash bucketing is uneven and the peak is what sizes a link, so do not quote the
mean.
THE GATE RUNS BEFORE THE MANIFEST IS READ, because the manifest is on the share
too. A bay that is not due must not touch the share at all - one read is cheap,
300 bays deciding to read in the same five minutes is the entire problem.
It FAILS CLOSED on an unparseable date. Failing open would restore exactly the
stampede this exists to prevent, and 30 GB cannot be un-sent, whereas a typo that
installs nothing says so in the log every cycle and is fixed in a minute.
A bay powered off during its wave installs on its next cycle instead. The wave is
an earliest-time, not a deadline, so nothing needs chasing afterwards.
-TestOnly reports a bay whose wave has not opened as COMPLIANT, because not
installed is genuinely its desired state today; DSC would otherwise call
SetScript every pass to be told to wait. -IgnoreWave is for proving a pilot bay
before opening anything.
Verified on Windows 11 build 26200, six paths: not-due installs nothing and exits
0; TestOnly while not due exits 0; a garbage date exits 1 having installed
nothing; -IgnoreWave installs against a future start; an opened wave installs;
and no wave arguments at all installs, which is what imaging needs.
ShopDB knew what a bay SHOULD have and nothing about what it DOES. Adding the
observed half makes a rollout a review instead of a typing exercise: the floor
reports itself in, you look, and you adopt.
The collection uses the mechanism that already exists rather than a new one.
POST /api/collector/printers dispatches to the printers plugin's
apply_collector_payload, the same ADR-006 hook the computers and backups plugins
implement. New client script, new plugin-owned table, no new transport and no new
credential.
OBSERVED AND ASSIGNED STAY APART, and that is the point rather than a detail. A
collector report can never write an assignment row: _reconcile_edges is the only
function that writes usesprinter/defaultprinter, it has two call sites, and both
are authenticated routes a human calls. If a drifted bay's own state were allowed
to become what it is told to install, every configuration error would become
permanent the next time that PC checked in.
Seeding an assignment from observed state is explicit -
POST /assignments/seed-from-observed - because a rollout adopts many machines at
once. It routes through the same _reconcile_edges as the editor, so there is one
write path with two doors, and a queue matching no known printer is REFUSED
rather than guessed into an assignment. That last rule is the lesson from the
measuring tools: adopting on a weak key produced 43 duplicate instruments.
Two fixes on top of what the agents built. The replace deleted a host's previous
rows by exact case-folded name while the read path treats a short name and its
FQDN as one machine, so a PC that changed spelling appeared to hold every queue
twice - which reads as drift that is not there. And the client sent 'reportedat'
where the declared schema said 'observedat'.
Also here: the legacy loader now imports machines.printerid, the classic system's
record of each machine's default printer, which it silently dropped - the
production import would have lost every one. And Set-ShopdbPrinters.ps1 finally
registers the per-user logon task, staging Apply-ShopdbDefaultPrinter.ps1 to
C:\ProgramData first because the share it lives on is mounted only during the
enforcement cycle and the task runs at logon when it is gone.
VALIDATED ON WINDOWS 11 (build 26200), not just on Linux pwsh, which parses these
scripts happily and executes none of the spooler branches.
The reporter: posts a correct payload with the X-API-Key header; resolves BaseUrl
and CollectorKey from HKLM when given no arguments; suppresses the virtual queues
by port; resolves port addresses; and reads the CONSOLE USER's default out of
HKU rather than SYSTEM's own, which is a different and usually wrong answer.
Two results matter more than the rest. With the spooler stopped, both the cmdlet
and the CIM path fail and the script posts NOTHING - verified against a capture
server that recorded zero requests, where an empty list would instead have
erased that host's observed rows and read as a bay that lost its printers. A
genuinely empty host still posts [], because that is a real and different fact.
The logon task registers as the Users group at Limited, and falls back to the
well-known SID S-1-5-32-545 when the group name will not resolve, as it will not
on localised Windows. It was then run with the source directory RENAMED AWAY, to
stand in for the share being unmounted, and it still moved the user's default -
which is the whole reason the script is staged to C:\ProgramData rather than run
from where it lives.
The guarantees against damage were re-checked rather than assumed: an empty
assignment changes nothing, an unreachable server changes nothing, -WhatIfOnly
leaves no queue, no task, no staged file and no registry value behind, and a
drifted queue is repointed IN PLACE with Set-Printer so whoever has it as their
default keeps it.
Not covered by any of this: the driver-staging path, which needs a real vendor
package rather than the class drivers a VM ships with.
A queue was matched by NAME alone, so a bay whose printer had moved, or whose
queue was built on a driver the site has since replaced, looked converged and
printed to the wrong device. Absence was fixed; drift was not.
Set-ShopdbPrinters.ps1 now repoints a queue whose port does not match the
address ShopDB holds, and swaps a queue built on the wrong driver. Both are
things ShopDB is authoritative about: where the printer IS, and what drives it.
CORRECTED IN PLACE with Set-Printer, never removed and recreated. The queue keeps
its name, its sharing, its permissions, and whoever has it as their default keeps
it - which is what makes this safe to run every cycle on a live floor. There is
still no removal code path in this script at all.
Two guards, because a repair that breaks a working printer is worse than drift:
the driver is only swapped when the wanted one is actually staged, and
-WhatIfOnly reports both kinds of correction without making either.
Verified on Windows against a queue that had the right name, the wrong port, the
wrong driver AND was the logged-on user's default: both fields were corrected and
the queue was still the default afterwards. The earlier no-op guarantees were
re-run and still hold - nothing assigned changes nothing, and an assignment with
no default leaves the user's own default alone.
The assignment belongs to the MACHINE, and until now there was no way to set it
except the generic relationships card or the API - the form for the thing the
feature is about did not exist. MachineForm now carries the picker, and PCForm
uses the SAME component rather than its own copy: the PC's set overrides the
machine's, and two implementations of that would drift, with the two ends of an
override disagreeing being exactly the bug nobody would spot.
The shared picker also fixes what PCForm did on save. It wrote row at a time
through the generic relationship endpoints, which is a non-atomic reconcile: an
HTTP failure part way left a PC half-assigned with nothing recording what was
meant. It now calls the reconcile endpoint, which validates the default before
writing anything.
A relationship type can now say it allows one active row per asset
(relationshiptypes.issingular, migration 7d34), and defaultprinter says it.
Cardinality belongs to the type rather than the printers plugin: core's create
path is where every hand-made link passes, and the next type meaning "exactly
one" gets the rule for free. Setting a second default REPLACES the first instead
of refusing, because "make this the default" means that - and a card answering
409 would leave the user hunting for the old row.
Without it the schema was happy to hold two defaults: the unique constraint is
(source, target, type), so two different targets are two valid rows, and the
resolver takes the OLDEST - the new default silently lost. Proven by disabling
the new rule and watching the tests fail.
FOUND WHILE TESTING IN A BROWSER, and it was not mine: MachineForm read
.data.data off computersApi.listAll(), which resolves to the ARRAY - fetchAllPages
has already unwrapped every page. The whole parallel load threw into the catch,
so every dropdown on the machine edit form came up empty and the machine's own
values never loaded. A build cannot see this; only opening the page can.
GET /api/printers/assignments/for-asset/<id> returns an asset's OWN assignment,
without inheritance, because the editor must show what this asset's rows say -
otherwise a machine's printers appear ticked on the PC that inherits them and
unticking one silently creates an override.
The last of the buildings-and-levels bugs, and the shop-floor clients brought
into the product.
Anyone on 0.11.0 through 0.11.2 should take this. Every hover mini-map in the
product reported "this asset has a position but no level" - the levelid prop
added in 0.11.0 was passed by none of its seven call sites - and the map PDF
printed markers from every floor onto one sheet, which nobody can correct once it
is carried onto the floor. The legacy import loader, still to run against
production, created markers with no level at all.
The gate that should have caught all three asked whether a FILE mentions levelid
rather than whether each position does. It now checks per occurrence.
Also: printers can be assigned to a MACHINE and reach whichever PC controls it,
so a reimaged bay reinstalls its own printers with nothing saved off the old PC;
printerdrivers can name a vendor, so two rows cover 41 of 44 printers instead of
twenty-one near-duplicates; and the collector reporter and EventSaver now live in
the repository with no site baked into either, the reporter generated per site on
request.
The client scripts were validated on Windows 11 against a live ShopDB, not only
by the suite: a bay with no rows of its own created both queues from its
machine's assignment, bound them to the right universal drivers, and set the
per-user default.
The version and the changelog are the release; the detail is in the entry.
Two rows now cover 41 of 44 printers. printerdrivers could only bind a driver to
ONE modelnumberid, so the HP and Xerox universal drivers - which between them
cover almost the whole floor - would have needed 21 near-duplicate rows pointing
at the same package. That is a table nobody keeps true, and it is why 42 of 44
printers resolved no driver at all.
printerdrivers gains vendorid, and resolution runs most-specific-first: the
printer's model, then its vendor, then the pre-vendorid convention of matching
the vendor word in the driver's name so a site that populated the table before
the column existed does not silently lose every driver on upgrade. A row that
names a vendor is never matched by its text, because a mis-set vendor resolving
to the wrong package is worse than resolving to none.
Six rows now resolve 44 of 44 printers at the reference site, and the DesignJet
correctly takes its own driver over the HP universal one.
Set-ShopdbPrinters.ps1 is the client half: ask for-host, create the queues that
are missing, record the desired default. It NEVER removes a queue - a bad minute
from the API must not take printers away from a working bay - and it never
fetches a driver, because downloading 48 MB while somebody waits to print is the
wrong moment. The common scope stages those.
Apply-ShopdbDefaultPrinter.ps1 applies the default in the USER's context, which
is the only context that can: SYSTEM cannot set a per-user default for somebody
else. It also turns off "Let Windows manage my default printer", without which
Windows silently overwrites the choice the next time anyone prints elsewhere -
a fix that undoes itself within a day.
VALIDATED ON WINDOWS 11 AGAINST A LIVE SHOPDB, not only by tests. Printers were
assigned to a MACHINE; a PC controlling it, holding no rows of its own, created
both queues bound to the right universal drivers, recorded the default and set
it, and a second run changed nothing. The first attempt failed with
"Relationship types are not seeded - run: flask seed reference-data", which is
the deployment trap the plan predicted, caught by an explicit error rather than
silently resolving nothing.
Printers belong to the bay, not to the box currently driving it. The assignment
goes on the MACHINE asset and reaches whichever PC controls it, so a reimaged or
swapped PC comes back with the right printers and nothing had to be saved off the
old one. The asset register is the backup.
New relationship type usesprinter ("this printer is installed here"), beside the
existing defaultprinter ("which of them is the default"), both seeded and both
given a propagation rail through controls. The rails are consumed at READ time
only: the create-time fan-out skips directional through-types, and controls is
directional, so assigning a printer to a machine does not copy rows onto its PC.
That is what keeps own-beats-inherited possible.
Resolution for a PC is its OWN rows if it has any, otherwise one hop out along
controls to the machines it drives. Whole set at a time, not merged: a PC with
its own assignment is overriding the bay deliberately, and the UI has to say so
or a tech "fixing" a bay by editing the PC will shadow the machine's record and
wonder why they keep disagreeing.
GET /api/printers/for-host/<hostname> is what the convergence client asks every
cycle. Resolved by hostname because the collector upserts PCs by hostname and an
office PC has no machine number. An unknown host, a site without the computers
plugin, and nothing assigned all return an empty set - that is the client's
designed no-op and it must stay indistinguishable from "assigned nothing".
PUT /api/printers/assignments/for-asset/<id> reconciles the whole set in one
call. The endpoint was specified, documented and asserted by three tests, and
never written - the verification pass caught that, with four failures. It
validates the default BEFORE any write, so a rejected request changes nothing;
soft-deletes rows that went away; and REACTIVATES soft-deleted rows rather than
inserting, because the unique constraint spans inactive rows and a blind insert
after an unassign raises IntegrityError on MySQL while passing on SQLite.
One default per asset, enforced here because the schema cannot: the constraint is
(source, target, type), which accepts two different defaults quite happily. Two
active defaults are still reachable through the generic relationships endpoint,
where the oldest silently wins - recorded in the proposal as the next thing to
close.
printerdrivers gains drivername: the exact string the INF declares, which
Add-PrinterDriver matches on and nothing else. Deriving it by parsing INFs on
hundreds of bays is fragile; a human confirming it once is not.
Assigning a printer to a bay is useless if the bay cannot install it, and the
fleet data says why that mattered: 42 of 44 printers could not resolve a driver.
This is the delivery half - the drivers themselves, staged once per bay, so that
creating a queue never waits on a download.
Install-ShopdbPrinterDriver.ps1 does one driver: trust the package's signer, then
pnputil /add-driver, then Add-PrinterDriver. Install-ShopdbPrinterDrivers.ps1
does a site's whole set from drivers.json, and answers a compliance question with
-TestOnly, which is what makes it a clean DSC Script resource rather than a
fire-and-forget install.
Deliberately SEPARATE from assignment. Drivers are large, near-identical across a
fleet and change rarely; assignments are small, per-bay and change often. Staging
the set in the GE-Enforce common scope means the assignment client only ever
creates a queue - it never fetches a 48 MB package while somebody is waiting to
print, or discovers the share is unmounted at the worst moment.
THE SIGNER TRUST STEP IS THE WHOLE TRICK, and it took a real driver to find it.
certutil -addstore on the .cat file satisfied the Xerox package and failed every
HP INF with "The publisher of an Authenticode(tm) signed catalog has not yet been
established as trusted" - a coin toss, not a mechanism. The certificate is now
extracted with Get-AuthenticodeSignature and added to Trusted Publishers, for
every catalog under the package rather than the first INF's neighbours. On a
locked bay there is no prompt to answer, so the old failure was silent.
Verified on Windows against real packages, not by reading: all six drivers this
site needs install through the script, a second run is a no-op, a wrong driver
name fails with the names the package actually offers, and the DSC cycle behaves
- TestOnly exits 1 on a clean box, install exits 0, TestOnly then exits 0.
The packages themselves stay out of git: they are licensed vendor binaries, and
they belong on the share beside the other imaging payloads.
DEPLOYING-DRIVERS.md carries the GE-Enforce entry, the DSC configuration and the
Intune shape, plus the constraint that has cost a session before: the SFLD share
is mounted only during the enforcement cycle, so this runs as a manifest entry
and never as its own scheduled task.
A site adopting ShopDB had to be handed two files and told what to edit in them.
Both are now the product's, and one of them the server writes for you.
GET /api/computers/client-script (admin) returns Report-AssetToShopDB.ps1 with
this site's values already in it: site_base_url becomes the -ApiUrl default and
the new computers_routableranges setting becomes -AllowedRanges. Only the
PARAMETER DEFAULTS are substituted - the copy in plugins/computers/client/ stays
runnable, so there is no second version to drift from the first - and everything
stamped stays overridable by argument or registry, because a bay may need to
differ from its site. Settings > Computers > Asset reporter edits the ranges,
downloads the script and shows its SHA-256.
The collector key is deliberately not stamped in, and a test fails if it ever
is. That file lands on every shop-floor PC, and a token spread across hundreds
of bays cannot be rotated quietly; it stays in the registry, provisioned per
ADOPTING-AT-ANOTHER-SITE.md.
The routable ranges are the last thing that was hardcoded in that script. They
are now a setting, so West Jefferson's two CIDRs move out of source code and
into that site's own configuration - which is what ADR-015 asks for - and a site
that sets nothing still works, because the script falls back to the NIC carrying
the default route.
EventSaver joins it in plugins/slides/client/, source only: EventSaver.cs and
EventSaver.ini, no compiled .scr - a binary is a release asset, like the
installer exe. The share path that was compiled into Config.Folder is gone. It
used to be the fallback when the ini was missing, which silently pointed a new
site at the reference site's file server; it is now empty, and failing visibly
beats displaying another site's slides. Verified by compiling the edited source
in the Windows VM with the in-box csc.exe: 15,872 bytes, exit 0.
Also: the DSC example in the adoption guide gains a CollectorRanges resource and
stops passing -ApiUrl to a script that already reads BaseUrl from the registry
the same example writes, and the guide points at the generated download instead
of hand-editing a URL.
The contract test caught the endpoint importing shopdb directly for the version
string, which ADR-002 forbids a plugin from doing. The product and contract
versions are in app.config now, which a plugin reads through current_app.
Adds docs/proposals/printer-assignment.md: assign printers to a PC in ShopDB and
let the bay install them, with what the fleet data says about drivers - HP and
Xerox cover 41 of 44 printers with universal drivers, there are no Brother
printers at all despite 208 files of Brother inkjet drivers in the installer,
and printerdrivers holds one row pointing at a per-model folder instead of a
universal driver.
Report-AssetToShopDB.ps1 lived on one site's imaging share and was, per the
adoption guide, "provided on request" - which is not a distribution mechanism for
a product meant to be adopted. It now lives in plugins/computers/client/, beside
the collector contract it implements (ADR-006), so the two version together.
Three things named West Jefferson and no longer do (ADR-015):
- The server. It resolves from HKLM:\SOFTWARE\GE\ShopDB BaseUrl - the value
Install-GEEnforce.ps1 already writes and the enforcement client cannot run
without - or from -ApiUrl. With neither it logs and exits 0 rather than
posting somewhere wrong. Any bay running this script runs the enforcement
client, so the value is present wherever it is deployed.
- The corporate VLANs. Two hardcoded CIDRs decided which NIC's address was
reported, with a comment reading "update if site re-VLANs". A site may now
name its ranges (-AllowedRanges, or a CollectorRanges registry value); with
none configured the NIC carrying the DEFAULT ROUTE is used, which expresses
"the routable NIC, not the controller NIC" without knowing any site's
addressing.
VERIFIED IN THE WINDOWS VM, not by reading it - and the VM earned its keep. The
local array was called $allowedRanges, which is the SAME VARIABLE as the [string]
parameter $AllowedRanges because PowerShell names are case-insensitive; the array
was silently coerced to an empty string, and .Count on a scalar string is 1. The
script therefore believed a range was configured, skipped the default-route
fallback, and reported no IP at all. Linux pwsh parsed it happily. Renamed to
$rangeList, and the four paths were then confirmed on Windows: no config skips
cleanly, BaseUrl resolves from the registry, an unconfigured site picks the
default-route NIC, a configured range selects or excludes as asked.
The hover mini-map said "This asset has a position (2835, 1410) but no level"
for every asset in the product. When 0.11.0 gave LocationMapTooltip a levelid
prop, NONE of its seven call sites were taught to pass one - printer, machine and
PC detail pages, the toner report, enforcement reports, the warranty chip and the
dashboard cards - so the component correctly reported a missing level and the
preview never drew. Two payloads behind those views also emitted mapx/mapy with
no level: the toner report and the enforcement report.
The map PDF export had the ORIGINAL bug still in it: it plotted every filtered
asset onto the sheet, so exporting the ground floor printed second-floor markers
on it. Worse than on screen, because nobody can correct a sheet once it has been
printed and carried onto the floor. It now exports only the level being viewed.
The legacy import loader sent mapleft/maptop with no level at three call sites.
That loader is the one still to run against production, and every marker it
created would have been undrawable. It now resolves the site's default level -
the legacy schema predates levels and has one floor plan, so that is what its
coordinates mean.
THE GATE MISSED ALL OF THIS because it asked whether a FILE mentions 'levelid',
not whether each position does: one module emitted 'mapx' six times and 'levelid'
once and passed. It now checks per occurrence, covers scripts/ as well as shopdb/
and plugins/, and fails any Vue file that binds tooltip coordinates without
:levelid. Both new rules were confirmed to fail the build against planted
violations before being relied on.
Printer QR labels: the asset number is no longer printed. A label now reads name
(8201-HPLaserJetPro), QR, FQDN, then IP. The name falls back to the assetnumber
because that is where sites actually keep it - every printer here has an empty
name field, so preferring the Windows queue name alone would have printed a blank
line on every label.
Bug fixes for the buildings-and-levels work in 0.11.0, every one of them found by
using the feature rather than by the suite.
Anyone running 0.11.0 or 0.11.1 with more than one level should take this
release: the map drew markers from every level on whichever floor plan happened
to be showing, which is the failure ADR-017 exists to prevent, and it was in the
one component that draws the map. The viewer also had no way to choose a level,
and the editor never accepted a click at all - its handler was bound only if the
map was already a picker at mount, and the editor opens with nothing selected.
Also: a level's native size could not be set (the settings page had no height
field), the same search could return different rows because fourteen searchers
truncated without an ORDER BY, and both map pages carried a scrollbar from
subtracting the wrong page chrome from the viewport height.
No schema change, and the plugin contract stays at 0.20.0.
The version and the changelog are the release; the detail is in the entry.
The previous commit stopped the panel header from colliding with the "Assets"
heading, but left the search box sharing a row with the type dropdown: 139px of
input for a placeholder needing 216px, so it still read "Search name o". A
control whose own label does not fit is not a narrower control, it is an
unlabelled one.
One control per row in a 318px panel. Measured in the browser: the input is now
286px against 216px of text, so the placeholder reads in full.
Clicking the map in the editor did nothing, ever. The click handler was bound
inside `if (props.pickerMode)` at mount, and the editor mounts with no asset
selected, so the handler was never attached; selecting an asset flipped the prop
but nothing rebound it. The asset forms were unaffected because their picker
mounts inside a modal that is already in picker mode, which is why this looked
like an editor-only fault. The handler is now bound unconditionally and
handleMapClick keeps its own picker-mode guard.
Verified in a browser against the dev instance, both ways: with the old binding
a click on a selected asset produced no position at all; with the fix the same
click reports 1652, 1138.
Markers are now drawn in picker mode too. Placing one relative to the machines
already on the floor is the entire task, and the old code skipped rendering them
whenever the map was a picker.
Two things that looked like stray widgets:
- The editor's panel header put a heading and three controls on one row inside a
320px panel, squeezing the search box until its placeholder read "Search na".
The heading takes its own row and the controls share the next.
- The legend drew its bar and border even with nothing to put in it, which read
as an empty input box under the toolbar. It renders only when it has entries.
Five defects, four of them mine from 0.11.0, found by using the feature.
THE SERIOUS ONE: ShopFloorMap never checked a marker's level. It skipped null
coordinates and drew everything else on whatever blueprint was displayed, so
level 1 markers appeared on level 2 - the exact failure ADR-017 exists to
prevent, in the one component that draws the map. The build gate did not catch it
because that rule checks files EMITTING mapx, not the component consuming it.
Markers are now filtered to the drawn level, and a position with no level is
omitted rather than approximated.
The map page had no way to choose a level at all. It read currentlevelid only to
title the PDF, so a second floor was unreachable from the viewer that most people
use. Adds a level selector (hidden when a site has one level), passes it to the
map, and switches the drawing, the bounds, the coordinate space and the markers
together - swapping the image without the bounds would place every marker against
the wrong scale.
Searching the map now follows results across levels: a search whose matches are
all on another floor showed an empty map while the filter counted them.
Floor map settings had NO height input - only width - so a level's native size
could not be set even while empty, which is the one time it is editable. Both
fields are there now, and size is editable on a level that has markers, because
refusing it blocked the case the feature was built for: a new blueprint of new
dimensions on a floor already full of markers. It confirms first and points at
landmark recalibration.
Search results differed between the sidebar box and the results-page box:
- 14 of 16 searchers truncated with .limit() and no ORDER BY, so the database
could return a DIFFERENT subset of matching rows for the same query. Every
searcher now ends in a total order (display key plus primary key).
- Searching a term already in the URL was a duplicate navigation the router
aborts, so the route watcher never fired and the button did nothing. The
sidebar never hit this, because it always navigates from another page - which
is why the two boxes appeared to disagree.
Also removes a scrollbar from both map pages. They subtracted 2rem and 40px from
100vh for the page chrome, which is really 90px of padding on .main-content, so
each overflowed by the difference. The padding is now a CSS variable both the
layout and the pages read. Measured in the browser before and after: 1058 vs a
1000px viewport, now 1000.
A patch release. The pagination-cap fixes, and the documentation that missed the
0.11.0 installer by a few minutes.
docs/UPGRADE.md is one of the documents the Windows installer puts on the server,
and its buildings-and-levels section was committed after 0.11.0 was tagged. So
0.11.0 shipped the page an operator reads WHILE upgrading without the guidance
about that upgrade - which matters most on an air-gapped box, where there is no
other way to reach it. The installer now also ships FLOOR-MAP.md for the same
reason the Windows runbooks ship.
No application code changed on the server side and the plugin contract stays at
0.20.0.
The version and the changelog are the release; the detail is in the entry.
Follow-up to the application-picker fix. Three of these were already wrong on
data that exists today, not merely latent.
Batch label printing is the worst of them: AssetLabelBatch asked for 500
machines or PCs, got 100, and printed a sheet that looked complete. With 262
machines and 290 PCs in the catalogue that is a physically short run with no
error anywhere - the operator finds out at the label printer, or later at the
bay with no label on it. PrinterQRBatch, USBLabelBatch and PrintedPartsLabels
had the same shape and are fixed alongside it, before their tables cross 100
too.
MachineForm's "controls" PC dropdown offered the first 100 of 290, so a
machine could not be linked to a PC sorting late in the list. NetworkDeviceForm
had it for models, which are already past 100 - and the same file already
called modelsApi.listAll() correctly two lines away.
Adds listAll() to the machines, computers, printers, network, measuring-tools,
USB and printed-parts APIs, all delegating to fetchAllPages().
Still outstanding: callers of vendors, locations, business units and the type
catalogues that ask for more than 100. Those tables are all well under the cap
today, so they are correct for now and wrong the day they are not.
get_pagination_params clamps perpage to MAX_PAGE_SIZE (100) and reports
nothing about having done so, so a caller asking for perpage: 1000 gets the
first 100 rows and a success response. Every picker built that way looked
complete and was not.
Found on a live site with 126 active applications: the 26 sorting last were
absent from the knowledge-base topic dropdown, so an article could not be
filed against them. Nothing was wrong with those application records, and
editing them could never have helped.
Adds fetchAllPages() to the api module, generalizing the one call site that
already handled this correctly (modelsApi.listAll), and points the four
application pickers at a new applicationsApi.listAll(): the KB article form,
the KB list's topic filter, the notification form, and the report filter
builder.
Lists that render a page at a time are untouched - they page for a reason.
Other callers still asking for more than 100 rows of vendors, locations,
models, subnets and the rest are latent: correct only while those tables stay
under 100, and silent on the day they do not.
The 0.11.0 release changed what a map position means and six documents still
described the model it replaced. Each of these could have caused a real mistake
rather than being merely out of date:
- IMPORT-API mapped legacy mapleft/maptop to mapx/mapy with no mention of the
level, so a scripted import - including the classic-ASP one still to run
against production - would have produced markers the map shows as "level
unknown". It now maps levelid too and says how to resolve the default level.
- API-REFERENCE enumerates the unauthenticated surface in full, because that is
what a deploy reviewer reads, and the three public /api/maplevels reads were
missing from it. Also records why the write split is asymmetric: repositioning
needs assets.edit, creating a level needs admin, since a level's dimensions are
the coordinate space every marker on it is expressed in.
- CONFIG still presented the four map_* settings as live, telling the reader to
re-upload a blueprint in a settings page that no longer drives the map. They
are marked superseded and kept for downgrade.
- UPGRADE gained a 0.11.0 section: nothing moves on screen, and replacing a
blueprint with one of different dimensions moves every marker on that level, so
recalibrate from landmarks rather than editing width and height.
- PLUGIN-HOOKS now states that a map overlay keys on assetid and must not return
coordinates or a level - a second copy of a position is one that can disagree.
Adds FLOOR-MAP.md, the operator's page: loading a plan, placing markers, and
what to do when the plan changes, with the reasoning left in ADR-017. START-HERE
routes to it from the new-site path, and specifically as the page to read BEFORE
a floor plan changes.
The floor map became a set of drawings instead of one picture, and search now
reaches what the forms have been collecting all along.
A site can hold more than one building and a building more than one level, and
every map position records which drawing its coordinates belong to. An existing
single-map site upgrades into one building and one default level carrying the
blueprint it already had, and nothing moves on screen. Landmark recalibration
moves a level's markers onto a redrawn plan, solved per axis from the landmarks
and never from the image dimensions, dry run by default, snapshotted before it
applies.
Two identifiers the UI collects were findable in almost no way and two plugins'
records in no way at all: gaugelabreference matched only measuring tools,
maintenancereference matched nothing anywhere, and USB devices and printed items
were absent from search entirely because neither is an asset. All four are
searchable now, from the global bar and from every list's own search box.
Contract 0.20.0 is additive: a plugin that writes a map position must write its
level with it.
The version and the changelog are the release; the detail is in the entry.
gaugelabreference and maintenancereference are collected on the form and shown
on the detail page for all five asset types, and no list's ?search= matched
either. Someone holding a physical tag could read it off the machine, type it
into the list they were already looking at, and get nothing back - while global
search, fixed in the previous commit, found it.
The clause is copy-pasted once per plugin, which is how all five came to omit
fields their own forms collect, so the test is one parametrized pass over all
five lists rather than five per-plugin tests that would drift the same way. It
also pins that widening the clause did not turn the box into a pass-through.
The api-inventory entries enumerate the fields each search matches, so all five
were stale the moment the queries changed; updated with the OpenAPI spec.
Not touched: the collector does not send either identifier, correctly - a
lab-assigned tag is not something an agent on a PC can discover. The legacy
import loader does not map them either, and the classic schema has no
gauge-named column to map from.
The map was one picture of one floor. A second floor was added, the blueprint
changed size, and machines moved, so a position now records WHICH DRAWING its
coordinates belong to.
Buildings and levels (ADR-017). Each level owns its blueprint per theme and its
own native pixel size; assets.mapx/mapy are pixels of assets.levelid, not of the
site. A position whose level is unknown renders "level unknown" and is never
drawn on the default level, because a marker on the wrong floor plan looks
entirely correct while pointing at the wrong place.
Repositioning in bulk: filter by unplaced, needs-review or level, search, place,
confirm. Landmark recalibration solves the transform PER AXIS from landmark
pairs and never from image dimensions - the canvas grew taller without
rescaling, so a dimension-derived scale would stretch Y by 1.57 and be wrong
everywhere. It defaults to a dry run, reports what would land off the drawing,
snapshots before applying, and clears mapverifiedat because a transform is a
guess awaiting review. Snapshots restore, including the level and the review
state, and a restore snapshots first so an undo is undoable.
Search: gaugelabreference was matched only for measuring tools and
maintenancereference was matched nowhere at all, for any asset type, while
Settings happily offers both identifiers on machines and PCs. A tag an operator
is told to record has to be findable or it is a write-only field. USB devices
and printed items were unreachable from search entirely - neither is an asset,
so the generic asset search could not see them and no searcher existed; they
now match on serial, asset tag, label, bin code and gage-lab tag, honouring
isactive, with Settings toggles and result labels to match.
The retired-application rule was half a rule: GET /api/knowledgebase hid
articles whose topic application is retired while global search still returned
them and printed the retired application as the subject. A filter is only real
if every path that reaches the row applies it.
Contract to 0.20.0 (additive): Asset gained levelid and mapverifiedat, Location
gained levelid, and resolve_asset_position returns the levelid belonging to
whichever source supplied the coordinates. The five plugins that write a map
position are re-pinned. The install-list text format gained levelid as a NINTH
field, appended, because the shipped Pascal installer reads fields 0-7 by index.
That installer still compiles in one drawing's dimensions and bundles one
blueprint, so its map is accurate for the default level only; /api/maplevels is
deliberately unauthenticated so it can read both at runtime once rebuilt.
Recorded in PRINTER-INSTALLER.md section 6 along with the other known gaps.
Migration 7d33 converts an existing single-map site into one building and one
default level carrying the old map_* settings, then assigns every placed asset
and location to it. Nothing moves on screen. Old settings rows are kept so a
rollback still finds them. Verified end to end on MySQL 5.6 from a
production-shaped database.
A privilege-escalation fix in the Windows installer, the multi-site portability
work, and a documentation and API surface that a second site can rely on.
The installer created C:\shopdb-flask, C:\Python314 and C:\MySQL84 without
breaking inheritance from the drive root, whose default DACL grants
Authenticated Users an inherit-only Modify. Two files inside those roots are
executed by someone else - shopdb-admin.ps1 self-elevates, and instance
config.py is loaded unconditionally by the app - so any logged-on user could
replace either and wait. Verified fixed as SYSTEM in the Windows 11 VM against
the shipped functions.
Contract 0.19.0 is breaking: get_dashboard_widgets returns data and shape, not a
component name. The change itself shipped earlier without a bump, with the
contract docs still teaching the dead shape; both are corrected.
The version and the changelog are the release; the detail is in the entry.
It claimed thirteen bundled plugins and listed thirteen; there are fifteen, and
the two missing ones - backups and tools - were undiscoverable from the first
page anyone reads. The count is gone rather than corrected, because a number
typed here goes stale exactly the way it just did; the generated map carries it.
The version and count guard now covers README.md and CONTRIBUTING.md, not just
docs/. A rule that leaves the most-read file in the repository outside it is
most of a rule.
Also names the reference site as the reference site rather than by plant, adds a
pointer to START-HERE, and corrects the distribution paragraph: it told a reader
of the public mirror that the code is only available on an internal server,
which is both wrong and unhelpful now that the installer ships as a release
asset.
The export gate now carries the site patterns, and three kinds of file tripped
it - two legitimately, one by construction.
Two test files held real internal subnets as fixtures. They are documentation
ranges now, which test the same logic and disclose nothing.
The project-map generator and a changelog entry named a file that is excluded
from publication, so a public reader was pointed at something they cannot see.
Both now describe what happened without naming it.
And the naming script has to CONTAIN the site patterns in order to grep for
them, so written literally the rule's own definition fails the gate that
enforces it. The patterns are assembled from fragments, the same trick the docs
publishability test already uses for the same reason. Verified the hard way: a
planted literal is still caught, so the fragmentation did not quietly turn the
rule into one that matches nothing - which is the obvious way for this fix to
have gone wrong.
DEPLOY-WINDOWS-IIS was a second copy of the manual IIS procedure that had
diverged from the first: a different MySQL version (8.0, which reached end of
life in April), a different port, a different plugin list, and a profile file
that does not exist. Two runbooks for one procedure means a reader follows
whichever they found, and one of them was wrong. INSTALL-WINDOWS-IIS covers
everything it did plus a preflight step and the subpath method, so the one
section it uniquely had - redeploying a hand-built server - is folded in there,
with the plugin-chain step it was missing and a note to back up first, and the
duplicate is gone. Everything that pointed at it now points at the survivor.
Three ADR statuses said something untrue.
ADR-013 said PROPOSED while half of it had shipped and ADR-014 had been accepted
on top of it. A decision that has been implemented and depended upon is not
proposed, and leaving one that way devalues every other status in the index. The
catalog half is still unbuilt, which is the ordinary state of an accepted
decision: accepted means settled, not delivered.
ADR-016 said ACCEPTED for a design where nothing is built - the endpoint and
permissions it describes do not exist, so a reader goes looking for them. The
status stands, because the decision does; the header now says so plainly and
points at where today's credentials actually live.
ADR-003 and ADR-004 were ACCEPTED with their own Decision lines still opening
"**PROPOSED:**", which reads as though the decision was never taken.
And the dashboard proposal carried Status: ACCEPTED, which belongs to a decision
record. A proposal is a proposal; the contract it produced is the ADR.
Four gaps a second site hits and cannot resolve by reading.
**Restoring on Windows** was one sentence - "the standard mysql < dump.sql" -
with no ordering. Restoring a database under running code that expects a
different schema turns a restore into a second incident, so the steps are now
ordered and each says why. It also says what `.env` costs if it is lost, which
is the part nobody discovers until they are already rebuilding: the dump does
not contain it, and without the JWT secrets every issued token dies, so every
collector and every GE-Enforce client on the fleet needs a new key.
**Rolling back** had a paragraph saying downgrades are refused and a backup is
the way back, but not the procedure. Rollback is restoring a matched pair, code
and the schema it expects, in that order - and the doc now separates it from the
case it gets confused with: a migration that failed mid-update has already been
rolled back by the installer, and fixing forward is the only move.
**Sizing, acquisition and support** were absent from the install guide entirely.
A reader could not learn how big a server to ask for, where the .exe comes from,
or where to raise a problem. The sizing is small and the reasons are stated, so
a site does not over-provision a VM for a load that is a few dozen users.
**Credentials** were described in three documents from three ends, so three
answers existed for where a key lives. One table, both ends - server and PC -
plus the two rules behind it: what a shop-floor PC holds is scoped to exactly
what it does, and a credential is delivered rather than typed, because a value
entered per machine is a value that is wrong on some machine.
The wiki had no entry point worth the name. Its Home page was hardcoded inside
the generator, had drifted into recommending INSTALL-WINDOWS-IIS - the manual
procedure these same docs tell you not to follow for a new site, because it
produces a server the installer then refuses to upgrade - and the page carrying
that warning, llms.txt, was never published at all, because the generator copied
only .md files. The adoption guide had zero inbound links.
START-HERE routes by what a reader came to do: stand up a site, deploy the
shop-floor tools, write a plugin, integrate with the API, understand a decision,
or fix something. It is the shortest correct path per role, not an index - the
sidebar is already the index.
FLEET-ARCHITECTURE is the page nothing else could assume. The server, GE-Enforce,
the asset reporter, the backup collectors and EventSaver were each documented
alone, each assuming a reader who already knew the other four. It draws the
shape, states the rule that explains most of the behaviour - nothing on the
server reaches out to a PC, the fleet asks - and ends with the table that says
which of the five programs to open for a given symptom, since knowing that is
most of the diagnosis.
wikigen renders START-HERE as the landing page rather than a list maintained in
a second place, and publishes llms.txt as LLM-GUIDE.
The inventory is hand-maintained, and should stay that way - its value is the
prose, and what an endpoint is FOR cannot be derived from the code. An audit of
all 372 documented operations found zero phantom routes and zero wrong parameter
names, so the maintenance is in good order.
What hand-maintenance cannot do is notice a route somebody added. Twenty-two
were undocumented: the entire backups plugin surface, every one of the dashboard
card endpoints added with contract 0.19.0, the GE-Enforce publish preflight, the
employee SSO resolver, the protocol update verbs, and the four /api/docs routes -
so the spec did not describe how to fetch the spec.
Coverage is now a test. It walks the live url_map and fails when a served route
has no entry, which means adding an endpoint includes describing it, in the same
commit, while the author still knows what it is for. The reverse direction is
checked too: a documented route that no longer exists sends a reader to a 404.
Writing that test found one more thing. The inventory writes multi-verb routes
as "PUT|PATCH", and neither the parity check nor the generator split on the pipe
- so those operations were absent from the published spec entirely, with nothing
reporting it. The spec now carries all 394 operations the code serves, which is
the first time the two numbers have matched.
The generator's own docstring claimed the inventory could be regenerated. It
cannot; nothing generates it. That sentence is why nobody noticed it was falling
behind.
The generated spec carried no `parameters` and no `requestBody` on any of its
372 operations. That is invalid OpenAPI 3.1, and the practical cost was worse
than the formal one: the MCP server builds its tools from this file, so every
tool had an empty input schema and silently dropped whatever the caller passed.
A request for one asset returned the list, and nothing anywhere reported an
error. All 118 templated paths now declare their path parameters, typed from the
Flask converter that named them, and write verbs declare a JSON body.
The body is an open object carrying the prose description rather than an
invented schema. The inventory describes bodies in sentences, and a field list
this generator guessed at would be worse than none - but "an object, described
here" is the difference between a client that can send a body and one that
cannot send anything.
Security was wrong on 123 operations. `jwt-optional` means "works logged out,
returns more logged in", which OpenAPI expresses as the empty requirement
alongside the scheme; publishing them as bearer-required told every reader that
a public endpoint needs a token.
Responses were one hardcoded 200, so a generated client had no idea a call could
fail. Every operation now documents the error envelope - and the envelope itself
is a defined schema, because its error nests under `data.error` rather than at
the top level, which is the single thing people get wrong when writing against
this API.
95 summaries were cut at 120 characters mid-word, which is what a tool picker
shows a user as the whole description of a call. They now end on a word.
Tests pin the shape rather than the prose. One of them contradicted an older
test that REQUIRED the contract version as a literal in PLUGIN-HOOKS.md - the
same copying that left nine documents stale - so that test now asserts the doc
points at the generated map instead.
Six procedures that could not be followed as written.
Eighty-nine curl examples single-quoted `Authorization: Bearer $TOKEN`, so the
shell never expanded it and the server answered 422 "Not enough segments". Nine
more did the same with X-API-Key. The other 129 examples in the same file
already used double quotes, so this was drift rather than a convention, and the
spec regenerated from it carried the fault onward.
The GE-Enforce report example put a `//` comment inside a JSON body. The server
parses with silent=True, so it saw `{}` and answered "hostname is required"
about a body that plainly has one - the worst kind of error message, one that
sends the reader to the wrong field entirely.
The IIS install ran `flask db upgrade` and a per-plugin install loop but never
`flask plugin upgrade-all`, leaving every plugin's own chain unapplied. That is
precisely the 1054 "Unknown column" a deploy then hits somewhere else, days
later, on the page that uses the new column.
The pilot runbook looped `flask plugin enable` over plugins that were not yet
installed; enable refuses those, so on a fresh database it exited 1 on every
iteration and enabled nothing. ADR-013 had already recorded that defect.
`apply-profile` installs and enables in dependency order, which is what the step
was reaching for.
DEPLOY-WINDOWS-IIS named a profile file that does not exist; the shipped ones do.
And PLUGIN-EXTERNAL-REPO never mentioned PLUGIN_TABLE_OWNERS, while the
migration engine raises for any plugin missing from it - so the guide's own
step 5 fails for any external plugin that owns a table. That the registry lives
in the framework repo is deliberate, so the guide now says so, and says what it
costs: a table-owning external plugin is a two-repository change, and a plugin
that owns no tables avoids it entirely.
Nine documents carried a hand-typed contract version and every one was stale.
One was load-bearing: PLUGIN-EXTERNAL-REPO.md told an external author to pin
">=0.13.0,<0.14.0" while the contract is at 0.19.0, so a plugin built by
following that guide is refused by the loader at startup. The plugin count was
wrong in six more.
They now point at docs/PROJECT-MAP.md, which is generated. A test enforces it:
no document may declare a version literal, a stated current version must match
the code, and a stated plugin count must match the tree. ADRs are exempt from
the current-version rule, because an ADR states the version a decision was taken
AT - that is a record of the past, and rewriting it would falsify the record
ADRs exist to keep.
CONTRACT-STABILITY.md was missing 0.17.0, 0.18.0 and 0.19.0 - including the only
BREAKING change in the series - in the one document a site reads to choose its
pin. All three are recorded, with 0.19.0 called out: it took something away, and
it shipped before it was written down, which is the argument for pinning tight
rather than trusting that a minor bump is safe.
The publishability gate caught internal tooling names and developer paths but
nothing site-specific, so roughly sixty leaks reached the wiki: the site name in
ten documents, real fleet hostnames in the collector and GE-Enforce examples, an
internal database name through the whole import guide, imaging-share paths, and
a maintainer's username as the Deciders line of every ADR and inside a generated
curl example.
None of it is a security matter on an air-gapped fleet. It matters because these
pages are read by engineers at other plants, and a document that names one site
throughout reads as that site's notes rather than a product's documentation -
which is exactly what it then gets treated as.
Examples now use neutral hostnames, the site is "the reference site" where the
distinction carries meaning, and ADRs are decided by "ShopDB maintainers". The
gate carries all of these patterns, so the next one fails a build.
Two documents leave docs/ because they were never written for an outside reader.
PROJECT-REVIEW.md is an internal health memo pinned to a commit from July, whose
headline finding (an untracked playbook) has since been fixed - it is history,
and git holds it. PILOT-DEPLOY.md is one site's own cutover runbook, complete
with a "re-measure before publishing" placeholder; it moves next to the loader
it belongs to, in scripts/site_imports/wjf/.
ADR-015 is AMENDED rather than rewritten. Its enforcement section still said
report-only and its backlog still listed hardcodes that are now cleared, which
left the record contradicting itself. The amendment says what changed and why
the report-only period ended; the original text stays, because what the decision
looked like when it was taken is the part worth keeping.
Also corrects llms.txt's response envelope, which had errors at the top level
and pagination at meta.total. Both are nested one deeper, so anything written
against that description read undefined on every error it tried to handle.
An ACCEPTED ADR ended with a stray tool-call fragment - it said nothing to a
reader and everything about how the document was produced, which is not what a
decision record is for. And the cutover playbook carried a real internal IP in a
file otherwise correctly scrubbed to documentation ranges. Both were published.
The publishability gate now catches all three shapes, so the next one fails a
build instead of reaching the wiki.
The adoption guide also told a site to copy two files that ship from nowhere:
the asset reporter and EventSaver are both on the reference site's imaging
share, not in this repository, and the guide never said so - it could not be
followed as written. It now states, per artifact, where each comes from, which
of them are pending a move into this repository, and that EventSaver can be
rebuilt from its one source file rather than taken on trust as a binary.
EventSaver was undocumented anywhere a reader could find it. What it is, the two
source modes and why HTTP is the one to use, how order.txt and the ShopDB feed
each decide the running order, the three command-line switches Windows calls it
with, where it must NOT run (a screensaver over a live dashboard or a metrology
run is a defect), how to build it with the in-box compiler, and a table of
symptoms with the first thing to check.
The adoption guide also stopped short of the question it invites: the fleet
tools assume GE-Enforce is already on the PC, so how does GE-Enforce get there.
Both real paths are now written down - imaging time, which is how the shop-floor
pipeline does it, and Intune, which is how the display cohort is deployed today
because those PCs are Entra-joined with no share. Machine Configuration is
possible and unshipped, with the caveat that matters: DSC and GE-Enforce are
both convergence loops, so give each a layer to own rather than pointing them at
the same configuration.
Three core pages and four plugin pages each imported qrcode and jsbarcode
directly, and each carried its own answer to the same questions: what margin,
what width, which error correction, how big a module must be before a scanner
can read it. The answers had already drifted - margin 0 in one place and 2 in
another, width 150 against 160 - and on a label that is the difference between
a sticker that scans and one that does not.
frontend/src/utils/codes.js owns it now: the label-stock presets, the quiet-zone
and margin defaults, CODE128 with no printed value, and the printer-resolution
arithmetic that only the Tech Tools generator had. A view passes what is
specific to its own label and nothing else - MachineBadge still asks for CODE39,
because the badge readers predate the shop-floor scanners and decode nothing
else, and that is exactly the kind of thing a call site should say out loud.
views/print/qrLogo.js is folded in rather than left as a second half-shared
helper that only some of the pages reached into.
The check script now fails a build that imports either library outside that
module. Without it this re-forks within a month: the next label page starts by
copying the nearest existing one, which is how it happened the first time.
Tests cover the part no amount of looking at a screen verifies - a QR that
looks fine at 96 dpi on a monitor can be unreadable at 203 dpi on half-inch
stock.
The asset reporter and EventSaver are both already built to be repointed - the
server URL, the API key and the targeting are parameters, an ini file and
manifest targeting, not code. Nothing said so, so the question "can another shop
use this" had no answer that did not involve reading PowerShell.
Worked examples for all three deployment paths, because sites have different
management planes and the choice is not ours to make: Intune (a remediation for
the reporter, a Win32 app for the screensaver, plus a Machine Configuration/DSC
form for estates already governed that way), a GE-Enforce manifest entry, and
manual installation for a pilot or a single bay.
The two traps are written down rather than left to be discovered. EventSaver
falls back to a path compiled into the binary when its ini is missing, and that
path belongs to the reference site - a missing ini is not a neutral default. And
a config enforced by hash reverts a hand edit on the next cycle, which is the
feature working correctly and reads exactly like a bug.
Also notes the reporter's -ApiUrl default still points at the reference site, so
every example passes it explicitly until that is fixed.
CLAUDE.md is read at the start of every session and claimed contract 0.16.0 and
1159 tests while the code was at 0.18.0 and 1567, along with a plugin count and
a chain head that were both wrong. Hand-written facts about a moving codebase
are wrong within a fortnight, and a reader who cannot tell which lines are stale
has to re-derive all of them.
So they are derived. scripts/gen_project_map.py reads the versions, the plugin
inventory, every Alembic chain head, the ADR index with status and the size of
the codebase out of the code, and --check fails a build where the committed map
no longer matches. CLAUDE.md now points at it, leads with the multi-site rule,
and lists the gates to run instead of restating the conventions they enforce.
The changelog's Unreleased section covered 8 of the 46 commits since 0.9.0 and
had no Changed, Fixed or Security sections at all. It now carries the whole
fortnight, including both contract bumps - which had never been recorded even
though ADR-002 makes contract versions their own series.
The scanner has been reporting the same count for weeks, which is what a rule
that only prints becomes. It now FAILS the build, and it looks where the leaks
actually were: PowerShell, the installer, the seeds, generated JSON, the
frontend - case-insensitively, across plugins, shopdb, scripts, deploy, tools.
A line that is deliberate declares itself with an ADR-015-OK marker and a
reason, so the claim is visible in review instead of tolerated in silence.
What it found, fixed here:
- The shadow client wrote one site's ShopDB URL into HKLM whenever the registry
disagreed. At the site it was written for that reads as healing drift;
anywhere else it overwrites the site's own address on every enforce cycle,
and the site cannot win because the cycle repeats. The bay's value now wins,
an explicit -BaseUrl seeds it, and with neither there is nothing honest to
write, so it says so and skips.
- The kiosk dispatcher fell back to one plant's host when HKLM was unset, so a
kiosk elsewhere quietly opened a server it has no business reaching. The
fallback is now this site's site_base_url, baked in at seed time, and the
dispatcher refuses rather than guessing when neither is set. Its legacy
shortcut matcher derives the host from that URL instead of naming one.
- The OpenAPI generator hardcoded a production hostname into every spec it
generated, which then published to a public wiki. The relative mount is the
only server it can honestly name; a site passes its own by environment.
- Placeholders and examples in the UI and the client help offered real internal
subnets and a real production URL. They now use documentation ranges.
Both publication gates - the export scrub and the docs publishability test -
carry the site patterns, which neither did. One plant's hostname, FQDN and
internal networks are out of the documentation and the generated specs.
Comments naming the reference site are reworded rather than deleted: the
reasoning is worth keeping, the plant name is not what makes it true.
get_setting on BasePlugin namespaces what it reads to plugin.backups.<key>,
while get_settings_defaults declares - and the settings page writes - the bare
key. So the retention read never found the operator's value and always fell back
to 0, and 0 means keep everything. Retention was configurable in the UI and did
nothing. It is the only place in the codebase using the namespaced helper.
The share root also stops shipping one site's file server as its default. That
put a site's internal topology in a bundled plugin and in the public mirror, and
pointed a second site at a server it cannot reach. Blank now, per ADR-015, and a
share kind with no configured root returns nothing rather than composing a path
from somebody else's hostname - a path built on an empty root is not a lesser
answer, it is a wrong one.
The publish gate exists because a minor version bump that NARROWS behaviour is
not backward compatible: _CmmVersion arrived in lib 2.6, and an older lib does
not know the field, so every gated entry looks unfiltered and it installs every
PC-DMIS version it cannot detect, on every CMM, in one cycle.
It was comparing the fleet's reported library versions against manifestversion.
That is the manifest's own 'Version' field. For a share-imported manifest the
two numbering schemes happen to coincide; for a scope authored in code they do
not, and seed_display_scope writes '2.0' - which every kiosk exceeds. So the
gate passed on the scope that most needed it.
A scope now declares minlibversion. Unset, the requirement is DERIVED from what
the manifest actually uses, so a scope written before this column existed is
still judged on its contents rather than on a number that says nothing about the
library. Only features that narrow behaviour belong in that table; an addition
an old lib ignores harmlessly needs no floor. manifestversion remains the last
fallback, which preserves what share-imported manifests already relied on.
Deleting a relationship is soft, so the row survives with isactive False, and
three things read them without knowing that.
Re-adding a deleted link answered 409 "this relationship already exists" about a
link the page no longer shows, and there was no way forward from the UI at all -
the row cannot simply be inserted again, since the triple is unique.
Reactivating IS the create for an inactive row.
The inverse guard blocked on a deleted inverse, which made "remove the existing
one first" - the instruction in its own message - fail to unblock anything.
fix-controls-direction retired the reversed row whenever a correctly-directed
one existed, without checking whether that one was itself deleted. So it removed
the only live link and reported a successful clean-up. It now reactivates the
row pointing the right way before retiring the one pointing the wrong way.
These are the commands the docs tell an operator to run against production.
An SVG is an XML document that may carry a script, and it is an accepted image
type because floor-plan maps and branding genuinely want vector. Loaded through
an img tag that script never runs, so the tiles and maps were never the risk.
Opening the file's own URL is - and the application image route is public, so
that URL needs no session.
Every route that serves an upload now goes through one helper that sends
Content-Security-Policy: default-src 'none'; sandbox, and nosniff. Seven routes
across core and five plugins, so a new one added later starts from the same
place rather than repeating the reasoning. Banning the format instead would
have cost the maps their only sensible one.
The app also sent no security headers at all. It now sets nosniff,
frame-ancestors self (as X-Frame-Options too, for the display bays' browsers)
and a referrer policy. Deliberately NOT a page-wide CSP: this serves an SPA with
inline styles, so a real script-src policy is a change worth making with the
frontend in front of you, and a permissive header claiming one would be worse
than having none.
Contract 0.19.0. send_upload is on the shopdb.api surface, because a plugin
serving user-supplied bytes should not have to remember these headers. The same
bump records that get_dashboard_widgets has taken data and shape rather than a
component name since the dashboard was rebuilt - that shipped without a bump,
while BasePlugin and PLUGIN-HOOKS.md both still documented the shape nothing
renders, which is how five plugins came to declare widgets pointing at
components nobody had written.
Minting derived a measuring tool's asset number from the HOSTNAME, so a
permanent instrument inherited the identity of whichever PC drove it that week:
replace the PC and either the number lies or a second tool appears for the same
physical unit. And because idempotency was keyed on the collector's own label,
it could not see a tool it had not created - on prod that left 43 legacy MT-####
tools shadowed by minted <HOST>-CMM twins, three records deep in places.
Resolution order is now most-stable-identity-first: the instrument named by
measuringtool-id.txt, then a prior collector link, then a tool this PC already
controls that somebody else created, then the reported machine number, and only
then mint. What minting produces should be read as a placeholder until a real
identifier is recorded.
Three separate 500 loops came out of the same mistake, looking a relationship up
by LABEL when assetrelationships is unique on (source, target, type):
- On a CMM the instrument IS the reported bay, so the machine sync has already
made a row for that exact triple - and it finds its own rows by that label.
Relabelling hid the link, so the next cycle built a second row for the same
triple and MySQL rejected it: 200 once, then 500 forever. The machine link is
now recognised and left alone; adoption only needs the identity.
- A part-marker PC hit it twice over, once on its partof row and once because
the marker's asset number is derived from the PC and could already be taken.
Both are get-or-create on the triple now, and an existing asset of that number
is adopted rather than duplicated.
A named instrument also supersedes a minted twin properly: the stale link is
archived by TARGET, not by object identity, which is what left a PC reading as
the controller of two instruments.
Reported identifiers are matched exactly rather than with ilike. They arrive
from a text file on a shopfloor PC, and ilike reads _ and % as wildcards, so
MT-600_ adopted MT-6001 and a bare % adopted whatever active asset came first.
A named id that is not a measuring tool is refused with a warning rather than
linked as one.
Core's Alembic env got connection.commit() when the stamp bug was found; the
per-plugin template did not. MySQL commits DDL implicitly, which flushes
everything queued before it including the previous migration's version stamp,
and the LAST migration of a run has no DDL after it - so its stamp rolled back
at close while its schema change survived. flask plugin upgrade-all then exited
0 having silently re-run that migration, and re-ran it again on every deploy
after. Invisible for exactly as long as every plugin head happened to be
idempotent.
Two were not.
backups 0003 cleared lastseenat for EVERY row, which is correct once and
destroys evidence on each repeat. It is now scoped to the backfill's actual
signature, COALESCE(collectedat, createdat) - the expression 0002 wrote - plus a
date bound. Both conditions are needed. Matching on collectedat alone misses
every row whose collectedat is NULL, so precisely the rows carrying the most
invented value would have kept it forever; and value equality is not a signature
on MySQL, where db.DateTime is second-precision and the collector writes both
stamps in one statement, so a genuinely fresh revision would read as a backfill
and be wiped. SQLite keeps microseconds, which is why no test could show it.
geenforce 0003 added a column unconditionally, so it failed on a fresh database
built from the models and on any re-run. Guarded like network0003prefix.
A directory created directly under C:\ inherits the drive root's DACL, and the
default carries an inherit-only Modify grant for Authenticated Users. So
C:\shopdb-flask, C:\Python314 and C:\MySQL84 were writable by every logged-on
user on the server.
That is a local privilege escalation here specifically, because two things this
installer puts inside those roots are executed by someone else: shopdb-admin.ps1
self-elevates with -Verb RunAs, and instance\config.py is loaded unconditionally
by the app through from_pyfile. Replace either, wait for an administrator or the
app pool to run it, and the code runs as them.
Hardening is applied at creation and RE-APPLIED on every run, so an upgrade over
an installation that predates this repairs the ACL rather than leaving the hole
in place. The MySQL data directory is locked down too: it holds the users table
and its password hashes, and it inherits ProgramData's read-for-Users.
The app pool is granted RX on the Python root alongside the app root. A venv's
python.exe is a copy but the DLL and the standard library are still read from
the base install, so without that grant the site 500s on every request once the
roots stop inheriting.
A Python installation this installer did not create is reported, not seized:
something else may depend on the ACL it has.
Verified as SYSTEM in the Windows 11 VM against the functions as shipped
(extracted by AST, not retyped): the inherited ACL really does grant non-admins
write; after hardening only Administrators and SYSTEM remain; a service identity
granted afterwards keeps access and later-created files inherit it; re-running
is a no-op; a missing path is silently skipped. 14 checks, all passing.
Every network device on this fleet already follows one convention, applied by
hand: AP-<name>, SW-<name>, SVR-<name>, IDF-<name>. 45 records, no exceptions.
The create form demanded the asset number anyway, so the same value was typed
twice and the convention held only as long as everyone remembered it.
The prefix now lives on the device type, and a blank asset number is generated
as <PREFIX>-<name>. Left explicit, an asset number always wins: a device
carrying a real identifier of its own - a vendor tag, a controller name, a
serial - keeps it. That is the platform rule, adopt where an identifier exists
and derive only where none does.
The prefix is NOT derived from the type name. "Access Point" and "Access Panel"
both initialise to AP, and assetnumber is unique, so the second type would
collide with the first on every device it created. It is nullable, so a type
that wants no prefix generates the bare name rather than needing one invented.
Names are sanitised before they reach a business key - the existing data
already shows why, with IDF-Telco-Demarc-#1 carrying a '#' into an identifier.
An existing prefix is never stacked: IDF-03 under type IDF stays IDF-03.
Test bays reported 0 installed / 0 skipped / 0 failed and nothing was being
processed. The share is a TRANSIENT mount: GE-Enforce.ps1 maps it for the length
of its cycle and unmounts at the end - mounted 12:10:02, unmounted 12:10:27 on
FB9TP7V3. The shadow task ran on its own 15-minute schedule, so it woke with the
drive gone, handed the engine a path that no longer resolved, and the summary
zero-filled. A silent nothing, indistinguishable from a healthy no-op, which is
why it read as "not processing" rather than as a failure.
Register-ShopdbShadow.ps1 is replaced by Invoke-ShopdbShadow.ps1, which runs AS
a manifest entry and invokes the runner directly. The share is mounted because
the enforce cycle is what called us; there is one cadence instead of two; and
there is no task to register, drift, or heal. It also unregisters the superseded
task, so bays that already carry it clean themselves up rather than keeping a
run that cannot see the share and overwrites the real result with 0/0/0.
The runner gains the Test-Path guard that should have been there: an unreachable
-ShareManifestPath now logs, writes an event, and reports a real failure naming
the reason, instead of running the engine against nothing. That silent zero is
what made this expensive to find.
Verified on the win11 VM with a substituted drive removed between runs - the
same call gives real counts while mapped and the guard when not. The earlier
task self-heal is moot now; it was a fix at the wrong layer, and testing against
a local path is what hid a mount I had already noted was dynamic.
A 3201dw is monitored and alerts correctly - Zabbix reports its cartridge names
and levels straight off the printer - but the Toner Report had no reorder chip
for it, because part numbers come only from modelsupplies and no family matched.
lookupsupplies returns [] for a model with no rows, so the printer showed levels
with nothing to order.
Adds the 218 family: 218A standard and 218X high yield in all four colours,
matched on 3201 and 3301 so the MFP siblings are covered by the same entry.
Part numbers and yields cross-checked against three independent sources rather
than inferred from the series - a wrong toner number means someone orders a
cartridge that does not fit, which is worse than the blank chip it replaces.
No drum or waste unit: none is listed as a separately orderable part for this
series, so the family is toners only.
"Checked 13 Aug 1:20 PM. Verified the backup taken 12 Aug 5:20 PM is still
current." Two facts, and one date could not carry both.
lastseenat moves on every successful confirmation and proves the check is still
running. collectedat moves only when the configuration actually CHANGED and says
what is stored. Showing lastseenat alone hid what is in the backup; showing
collectedat alone read as "last backup was a month ago" on a machine that is
perfectly healthy, because a stable config writes no new revision. The hover now
states the check and the capture separately and says outright that the second
being older is the normal case.
The stale wording gets the same treatment: it names the check that stopped AND
the newest copy that exists, which is the thing someone recovering from it
actually needs.
collectedat is exposed as backupcollectedat. Both stay tooltip-only - the badge
is still just the kind and a colour, so nothing here changes what a new backup
kind has to do to inherit it.
Two fixes to the same table, in the same regions of the same files.
ASSET LINK POINTED AT THE WRONG RECORD. The Asset chip linked
/machines/<assetid>, but /machines/:id keys on machineid - the plugin extension
id - as MachineDetail itself does everywhere. So the link landed on whichever
machine happened to carry that number: a wrong page that looks right, which is
worse than a 404. Same for /measuringtools/. The API now returns
machinepluginid / toolpluginid beside the asset ids and the view links on those.
Both lookups are import-guarded, and with no plugin id the number renders as
plain text rather than a link that misleads. AssetRelationships already resolved
this correctly; this brings the reports table in line.
BACKUP COLUMN READ AS NEGLECT. It showed a raw date, and a revision is only
written when the config CHANGES - dedup means a machine stable for months has a
months-old newest revision and is perfectly healthy. The column already used
lastseenat, the last time the collector CONFIRMED the config, but a bare
timestamp says "nothing has happened since", which at the default 24h collection
interval IS the healthy steady state. It made a working system look stalled and
made the reader do arithmetic against a setting they would have to go and find.
It now returns backupok and shows a badge naming the kind, green when confirmed
recently, red when not, with the date and an explanation in the hover. backupok
is tri-state on purpose: null means no revision at all, and renders as NO badge
rather than a green one, because "never seen" must not read as healthy. The
threshold is the backups plugin's own backups_staledays, read through its
service so there is one definition of stale rather than a second drifting here.
Nothing in the badge is kind-specific, so a backup kind added later inherits it
by existing. docs/BACKUP-KINDS.md records that, the BackupKind contract, and why
the rule is time-based rather than per-kind.
Registered fine and never fired once. Three faults, all found on the win11 VM.
A `-Once -At (Get-Date)` trigger does NOT run immediately: its first run is the
start boundary PLUS the repetition interval, 15 minutes out. This ships as a
DetectionMethod=Always entry, so it ran every enforce cycle, 5 minutes apart,
and each Register-ScheduledTask -Force reset the start boundary to "now" -
pushing the first run back to +15 before the previous +15 could elapse. 5 < 15,
so the task sat Ready at LastTaskResult 267011 (SCHED_S_TASK_HAS_NOT_RUN)
forever. It now registers only when the task is missing or its arguments
changed, and starts it once on first registration rather than waiting out the
first interval.
A bay provisioned by the broken version carries a task with correct arguments
that has never run, so "leave it alone if it matches" would have stranded
exactly the machines that hit the bug. If the task has never run it is kicked
once; after that LastRunTime is set and the check is a no-op.
None of this was visible. The engine records only "ps1: <path>" and an exit
code for a PS1 entry, so Write-Host reached nothing, and with the fail-safe
`exit 0` on every path a silent early-out was indistinguishable from success.
It now also writes C:\Logs\Shopfloor\shadow-setup-<date>.log.
Scope is no longer hardcoded either: this script is shipped by more than one
scope now, and a wrong value would shadow the wrong manifest silently. It is
derived from the script's own directory, the same way the share manifest path
already was, so the two cannot disagree.
The report answers a purchasing question, and it was answering it in seven
columns, two tables and a rowspan. What someone actually needs from it is a
short list of what to buy.
So it opens with that list, grouped by part number with a quantity. Two
cartridges of the same part in different printers is a quantity of two, which
is the number an order needs and the one a per-printer table made the reader
count by hand. It covers what is empty plus what goes within a fortnight -
ordering only what is already empty means running empty. There is a copy
button, because it ends up pasted into a mail.
Below it the cartridges sit in urgency bands rather than in one long list
sorted by a number. The question is which pile a thing is in, and a pile that
is empty is worth seeing as empty. Everything past "empty" starts collapsed;
the order list above already covers the same ground in a tenth of the height.
The row is a cartridge now, not a printer, so it can carry its own part number,
its own level bar and its own countdown. Nesting supplies under a printer meant
opening a printer to find out whether anything on it needed doing.
Cartridges with no part mapped are counted on a single line rather than given
one each. They cannot be dropped, since that would quietly shorten the order,
and they cannot be ordered from here either - the job they represent is
mapping them, which is one job however many there are.
Bands and the order horizon are decided server-side, next to the arithmetic
that produces them, so a heading cannot disagree with what got added to the
list.
Checked against a fleet of 43 dev printers with real part mappings, driven by
a stub Zabbix - live Zabbix is not reachable from the dev box.
Prod grew rows saying both "PC controls 2005" and "2005 controls PC", and a CMM
PC showing "<- controls from CMM4" beside its own outgoing link. Only one
direction can be true: a PC drives a machine, never the reverse.
Nothing stopped it. The duplicate check was keyed on (source, target, type), so
the inverse inserted cleanly, and the Add Relationship dialog offers an incoming
direction that writes exactly that. The legacy import stores controls the wrong
way round as well. Directional creates now refuse the reverse with a 409 naming
the row that already holds it, and refuse self-links, which render as a
duplicate on the asset's own page and mean nothing. Symmetric types are exempt:
Dualpath stores both directions on purpose and the card collapses them. The
propagation fan-out got the same guard so a rail meant to spread one direction
across sibling bays cannot manufacture a pair.
fix-controls-direction only matched source assettype 'machine', so every
measuring_tool, printer and network_device row it was written to clean survived
it - which is why running it would never have fixed the CMM. It now matches any
non-computer controlled BY a computer.
New `flask relationships audit` reports what is already stored: reciprocal
pairs, self-links, and PCs controlling several assets of one type. Read-only,
and it prints each row's label because that usually names the writer outright -
collector:* means this code made it, anything else means a person or the import
did. That distinction decides the fix for duplicate device assets, which is not
in this commit: the collector keys idempotency on its own label, so a device
somebody created by hand is invisible to it and it mints another, and the
adoption rule needs the audit run against prod before it can be written.
Two false positives were found writing it, against the dev database, and both
would have made the report useless. A self-link is its own inverse, so it was
counted as a reciprocal pair AND printed twice. And Dualpath siblings looked
like duplicate devices - a dual-bay machine is one physical machine with one
controller and controls is propagated to both bays deliberately. That was 30 of
32 findings, consecutive bay numbers pair by pair.
The cheap cutover the doc described - manifest over API, payloads still on the
share - does not work. Measured on the win11 VM against lib 2.6 with the same
manifest and engine, varying only InstallerRoot: the payload cache gives
0 installed / 2 failed ("CMD/BAT not found" under the cache), the scope
directory gives 2 installed / 0 failed.
Two causes. Invoke-ShopdbEnforce.ps1 takes no -InstallerRoot and hardcodes the
payload cache outside shadow mode, which is right for displays where every
entry is inline and wrong for every smb entry. And -IncludeCommon cannot work
for smb payloads at all: they live under their own scope directory, so a merged
manifest needs two roots where the engine takes one, and Join-InstallerPath
rejects rooted values by design so absolute paths are not a way out. The real
dispatcher never merges - it calls the engine once per scope with that scope's
root - and a fix should follow that shape.
Both claims are corrected in the doc rather than deleted, because reading it
was what made this look like a twenty-minute job; it would have failed 29 of 31
entries on the pilot bay.
Shadow mode is unaffected - it points InstallerRoot at the share scope dir,
which is the working path - so the pilot proceeds. Register-ShopdbShadow.ps1
is the reference copy of the script that puts a share-attached bay into shadow:
it writes BaseUrl, then registers the task as SYSTEM. It derives the share
manifest from $PSScriptRoot rather than hardcoding a drive, because GE-Enforce
mounts the share to a letter it picks at run time and SYSTEM has no standing
UNC access. Delivered as a manifest entry gated to one hostname, so the pilot
needs no hands on the bay and heals if it drifts.
Four defects stacked into one nonsense report: cartridges at 20% claiming four
days, cartridges at 1% claiming weeks.
The root cause is a Zabbix API detail. `limit` caps the whole result set rather
than each item, and the query sorted ascending, so the cap kept the OLDEST rows
in the window. A four-cartridge printer polled every five minutes writes over
100k readings in 90 days; the forecast was fitted to the first few days of that
and nothing since. Every rate was real and every rate described a cartridge
thrown away three months ago. Nothing in the output looks wrong, which is why
it needed pinning in a test rather than a comment.
A 90-day burn rate does not need every individual poll, so a long window now
reads hourly trends - the table meant for this, a tenth of the rows, and kept
longer. Raw history serves short windows and any item a site keeps no trends
for. Both are fetched newest-first with the budget scaled per item.
Second, the countdown was computed from the last stored reading while the level
displayed was the live one, so the two could disagree by a whole cartridge. The
live level is now what the countdown divides. A live level far above the stored
run means it was swapped since the last reading, and that is reported as a
replacement rather than as a collapse in the burn rate.
Third, at or below 5% a cartridge reads as empty rather than as a slow drain.
At 1% losing a tenth of a point a day the arithmetic says ten days. The printer
is out of toner, and it is the first thing to order.
Fourth, the days-left column spanned the printer's rows, so the printer's
soonest figure was printed beside every supply it had. That alone accounts for
the shape of both complaints: a healthy cartridge wearing its neighbour's
deadline, and an empty one wearing a number that belonged to nothing on its row.
Also fixes float-typed supplies vanishing from any printer that also had an
integer-typed one - they live in different history tables and the fetch stopped
at whichever answered first.
Not verified against live data: Zabbix is not reachable from the dev box.
The fleet table has two time columns and staleness only looked at one. A PC can
go quiet in either direction: silent to the server, or still posting while its
own cycle has stopped advancing. Only the first was caught, so a machine whose
Last check-in had not moved since the previous morning still showed 'ok'.
The two are tested independently rather than by taking the newer of them.
receivedat is the server's own record and cannot be argued with; lastcheckin is
asserted by the client, so a wrong clock there must not be able to vouch for a
PC the server has not actually heard from. Checking them separately means a
future-dated lastcheckin cannot mask real silence.
The tooltip now names both times, since which one stopped says what kind of
failure it is.
A report records how ONE cycle went. Nothing ages it, so a PC that stops
reporting keeps the status of its last good cycle: switch a machine off after a
clean run and it shows 'ok' indefinitely. One had been offline more than a day
and still read 'ok'.
Silence is a different fact from the last cycle's outcome, so it is computed
separately rather than by rewriting the stored status. receivedat - the server's
own clock, not anything a client asserts - is compared against
geenforce_reportstaleminutes, default 30, which is roughly two missed cycles at
the usual cadence. Set it to 0 to turn the check off.
In the table 'stale' takes the badge, because a status from a machine that has
not spoken since is not evidence of anything. What it last reported stays in the
tooltip with the time it was heard. The stored status is untouched: it is still
a true record of that cycle, just not proof the PC is alive.
A site whose scope enforces less often than the threshold will read stale while
healthy, which is what the setting is for.
Core called the roles dashboard / lobby / partskiosk. The kiosks call them
Dashboard / Lobby / 3DPrintRoom, which are the literal contents of
C:\Enrollment\display-type.txt, read by the GE-Enforce dispatcher to pick a
target. Two vocabularies for three kiosks, each with its own copy of the same
route map.
That is not cosmetic. A display reporting its own type sends what its file
says, so it could report a role core would not accept, and core could store
'partskiosk', a value no dispatcher would ever match. The enforcement report
column would have shown one vocabulary from the device and the other from the
DashboardDefault fallback, in the same column.
The machine's file wins, because that is what a person edits. DISPLAY_ROLE_PATHS
takes the kiosk spelling and the display scope now uses that dict rather than
holding a second one, so the two cannot drift again. normalize_display_role
resolves any casing and the retired 'partskiosk' forward; the dispatcher already
matched its map case-insensitively and the server now agrees with it.
Nothing is turned away over a capital: the API accepts any spelling and stores
the canonical one, displaypath resolves through the normalizer so rows written
before this keep working, and the settings dropdown canonicalises on open so an
old value does not render as a blank select.
A reported subtype is normalised on the way in, but an UNRECOGNISED one is kept
verbatim. That is a kiosk with a typo in its file or a role nobody declared, and
both are worth seeing in the fleet table rather than blanked or guessed at.
Contract bumped for the added names. DashboardDefault is finally listed in
__all__ too - 0.17.0 put it on the surface and never exported it.
SQLAlchemy 2.0 connections do not autocommit, and on MySQL alembic reports
"non-transactional DDL", so context.begin_transaction() is a no-op. Nothing in
a migration run committed.
It looked like it worked because MySQL implicitly commits on DDL: every
ALTER/CREATE flushed whatever was queued before it, including the PREVIOUS
migration's version stamp. The LAST migration of a run has no DDL after it, so
its stamp was rolled back when the connection closed. `flask db upgrade` then
exited 0 with the schema change really applied and alembic_version one revision
behind, and re-ran that same migration on the next deploy. A migration that is
not idempotent applies twice.
Found while adding a core migration: the column default really changed and
alembic_version still named its parent, with UPDATE alembic_version followed
immediately by ROLLBACK in the log.
Existing databases upgraded before this are one revision behind their real
schema. Compare `flask db current` against `flask db heads`; re-running the
upgrade re-applies the final migration once and lands the stamp.
Install-ShopdbKiosk.ps1 lays the enforce client down once at bootstrap and
never refreshes it. So a client change rode the code deploy to the server
and then sat one directory away from where kiosks actually fetch, waiting
for someone to re-stage the installer bundle by hand - which is how the new
display-type reporting reached prod and changed nothing on any kiosk.
The module now ships as a manifest entry like everything else in this
scope: inline over HTTPS, Hash detection against the exact bytes shipped,
written to the same path the installer uses so bootstrap and self-update
cannot disagree. Ordered first, so a stale client refreshes before anything
leans on it. The installer keeps its real job - a fresh kiosk still needs
something that can talk to shopdb - it just stops being the update path.
Self-modifying by design: this module is what stages payloads, but
PowerShell loads it into memory at start, so rewriting the file mid-run is
harmless and lands on the next cycle. Pilot a client change on ONE kiosk
before the fleet: a broken module cannot fetch its own replacement, and on
a share-less display that means a site visit.
A display knows whether it is a Dashboard, a Lobby screen or the 3D print
room - the dispatcher reads C:\Enrollment\display-type.txt to choose which
page to open. It never told shopdb, so the fleet table inferred it from the
DashboardDefault fqdn mapping, which is empty unless somebody added a row
per kiosk. The column was blank for every display.
The client now reads that file and reports it, the report stores it, and
the API prefers the reported value with the old mapping left as a fallback
for hosts still on an older client. Reported by the device beats inferred
from a lookup table, the same way enforcerversion already works. A PC with
no display-type.txt reports nothing rather than something invented, and an
empty string lands as NULL.
Two guards had to learn about it. The DDL parity check read only the 0001
baseline, so a column added by a later revision looked like drift even
though its migration existed; it now runs the whole chain, which is what
'do the models match what the migrations build' means. 0002 added a whole
table rather than a column, which is why this is the first time it bit.
Two caps, one inside the other: the section shell at 1400px and the reports
view at 1100px. Fine for nine columns; the reports table now carries twelve,
including the asset a PC drives, its location and its backup state, so it
was being squeezed while empty page sat to the right. Both are gone, and
the manifests editor gets the width too.
The Asset column no longer falls back to the PC's own asset number. The
collector stores a PC's hostname AS its assetnumber, so for any PC that
drives nothing that column just repeated the Host column next to it. It now
shows a dash, and carries content only when it says something Host does
not: the machine a bay PC controls, the tool a measuring PC controls, or
the role of a display.
The reports table 500'd on every load: the backup lookup ordered with
ORDER BY lastseenat DESC NULLS LAST, which SQLite accepts and MySQL
rejects outright. Every test passed and the real database refused the
query - the tests run on SQLite, so the dialect difference was invisible.
Sorting in Python removes the dependency for nothing: the rows are one per
host per kind. The regression test pins which revision wins, including
that one never confirmed does not, and says why the sort lives here so it
does not get helpfully moved back into SQL.
The enforcement reports needed to name what a display IS, and reached
straight into shopdb.core.models.dashboarddefault to do it. Plugins may
only touch core through shopdb.api, and the contract test said so.
The role belongs on the surface rather than behind it: it lives in core,
no plugin owns it, and a plugin reporting on displays has no other way to
resolve it. Added there and the version bumped, which the docs test pins.
A report row carried a hostname and some counts. Everything an operator
wants next hangs off the asset behind that host, and none of it was there.
Host now links to its PC page. Beside it, what the PC IS or DRIVES: the
machine number for a bay or part-marker PC, the measuring tool for a tool
PC, the role for a display - each linking to that asset's own page. A map
pin appears only when the asset has coordinates and hovers to the same
floor-plan preview the asset pages use; an icon that opened an empty map
would be worse than no icon.
The backup column is LAST CONFIRMED, not last changed. Dedup means an
unchanged config writes no revision, so a machine stable for six months has
a six-month-old newest revision and is perfectly healthy - the one to worry
about is the machine whose backup stopped running. It reads lastseenat and
names the kind rather than assuming ntlars, since udc/file kinds on the
share are coming.
Resolution is bulk, never per row: this table shows the whole fleet, so a
lookup inside the loop would be one query per PC. It reads the collector's
existing 'controls' relationship rather than re-deriving which machine a PC
drives - that same resolution living in two places is what put a wrong
subtype filter on the map. Every plugin it touches is optional, so each
lookup is ImportError-guarded and a lean build renders the table without
those columns. A host ShopDB has no asset for still shows: the enforcement
result is real even when the inventory is behind.
Every column in the entries table was sized to its button text with nothing
spare, so with flex-wrap on the action cell the Edit and Delete buttons
wrapped out of their 128px column and sat on top of the entry description.
The order column was tighter still, and its Up/Down buttons carried almost
no padding, which made them hard to hit as well as hard to read.
Widened the order, type and action columns to fit their contents on one
line, stopped the action cell wrapping, gave the buttons real padding, and
added vertical cell padding with top alignment so a two-line entry no
longer runs into the row beneath it.
The engine treats a minor-newer manifest as backward compatible and carries
on. That holds for additions which WIDEN behaviour - an old lib skips a Type
it does not know - and inverts for one that NARROWS it. _CmmVersion arrived
in lib 2.6 as a minor bump, so a PC on 2.5 does not recognise the field,
reads every gated entry as unfiltered, and installs every PC-DMIS version it
cannot detect, on every CMM, within one cycle.
The share runbook already says push the lib first. A runbook is not a
control, and the failure is silent, fleet-wide and about five minutes fast.
ShopDB already had the evidence and was not using it: every enforcement
report carries the enforcer version, and publish_scope had no gate at all.
It now compares the scope's manifest version against the versions PCs
actually report for that scope and refuses when any is behind, naming the
hosts. force=True for someone who knows why. A report with no or an
unreadable version counts as behind - that field arrived with the
summary-emitting engine, so its absence IS an old lib, and treating unknown
as safe is precisely how this fails open.
A scope nobody has reported for still publishes, or a fresh site could
never publish anything. Versions compare numerically, since as text '2.10'
sorts below '2.9'.
Also exposed as a preflight endpoint so the UI can warn before someone
clicks publish, and as a 409 with the offending hosts rather than a 500.
Every kiosk was sitting on Edge's restart-to-update prompt. The scope
already sets RelaunchNotification=2 so Edge restarts unattended, but
RelaunchWindow defers that restart to 02:00-04:00, so during the day the
update waits and there is nobody on site to dismiss the prompt.
A one-shot entry runs the Edge updater and stops the browser. It does not
relaunch it - the enforce task is SYSTEM in session 0, where a launched
browser is invisible - so it leans on the watchdog that already relaunches
the kiosk from the Startup shortcut. That relaunch is what applies the
staged update.
One-shot is DetectionMethod=MarkerFile. The engine writes the marker only
after a 0 exit, so a failed run retries next cycle instead of being
recorded as done, and the script exits 0 when no Edge was running - that
is a success, and failing it would withhold the marker and re-kill Edge on
every cycle from then on. The marker path carries a date, which is the
re-arm mechanism for a future update.
Ordered after the watchdog entry: a display seeing both for the first time
must have its relauncher registered before anything stops the browser.
The publication rsync excluded 'tools' to keep the repo-root tools/ dir
off GitHub. rsync patterns without a leading slash match at any depth, so
naming a plugin 'tools' meant plugins/tools/ was dropped too. It was
committed, exported, bundled and deployed, and the first sign of trouble
was `plugin install tools` on the server failing on a manifest.json that
had never been shipped.
The root-intended excludes are anchored, and the sync now verifies that
every plugins/*/manifest.json in the working tree came out the other side.
A silent omission of a whole plugin should not be something a deploy
discovers for us.
Ten routes shipped over the last few commits without reaching the inventory,
which is the file everything else is generated from - openapi.json, the /api/docs
browser, and the per-site build. An endpoint absent from it is invisible to
anyone integrating against the API even though it answers requests.
The auth notes matter more than the paths. API-REFERENCE tells a deployment
reviewer that essentially every GET is optional-auth; two of these are not, and
an exception buried in a table of 372 operations is one nobody reads. The
installer download and the warranty proof are called out with why: one is
licensed vendor software, the other carries pricing and a service tag. The
application image is listed with the fully public endpoints, since tiles render
before login.
The classic ASP site prints a week number under the site title, and people
quote it in conversation and on paperwork. Anyone with both sites open needs
the two to agree, so this is a port of the old arithmetic rather than a fresh
interpretation of what a fiscal week is: ISO 8601, week 1 contains 4 January,
and the week's Thursday decides which year it belongs to. That rule is what
makes late December and early January land in the right year, which is exactly
where a naive day-of-year count goes wrong, so it is what the tests cover.
Worth recording: a true GE fiscal calendar need not follow ISO weeks. Nobody
has asked for a different rule, and inventing one here would silently disagree
with the site people compare against.
Computed in local time on purpose. The number people quote is the one on the
wall where they stand, and a UTC week rolls over hours early in the evening at
a US site. The sidebar re-checks every half hour; the shop-floor board picks it
up with the clock it already ticks.
The toner report says what is empty now. It could not say what to order, and
nothing recorded how fast anything drains - every level read was cached for
five minutes and then discarded.
Zabbix has been keeping the history all along; we simply never asked. One
history.get gives both answers, because a cartridge only goes DOWN while it is
in use: a rise is a replacement. Count the rises and you have how many
cartridges a printer has been through; fit a slope to the readings SINCE the
last rise and you have days-to-empty. Fitting across a replacement averages a
spent cartridge with a fresh one and describes neither.
Sorted by days left, which is the point. A cartridge at 60% dropping 5% a day
needs ordering before one sitting at 8% that has not moved in months, and a
level-sorted list ranks those backwards.
It refuses to guess. Too few readings, a level that has not moved enough - many
printers report in 10% steps and sit on a plateau for a fortnight - or a recent
replacement each produce no estimate and say which. Those printers are listed
separately rather than sorted in as 0 or as 999, since a printer without an
estimate is neither urgent nor safe. Estimates show what they rest on, because
"9 days from 21 days of readings" and "9 days from 2 readings" are not the same
claim.
A separate report card, not an extension of the toner report: that one is an
exceptions list a tech acts on today, this is an ordering view read monthly,
and the history query is heavier than the live read it would have slowed down.
The analysis is pure arithmetic over a list of readings, so the 14 tests cover
the noise wobble, the plateau, the swap, junk rows and division by zero without
needing Zabbix. Zabbix being unreachable is reported as such rather than
rendering an empty table that reads as "nothing is due".
A provider lookup answers whether a unit is covered. It does not produce the
invoice or the extended-warranty certificate, and a manually entered warranty
had nowhere to keep one - so the proof stayed in somebody's mailbox until they
left.
Two columns rather than one: the served URL of the stored document, and the
name the vendor sent it under, because "Dell invoice 4471.pdf" is what a person
recognises a year later and "warranty-12.pdf" is not. The download route sends
the original name back.
Authenticated in both directions, unlike an asset photo: an invoice carries
pricing and a service tag. One document per warranty, replacing any prior
extension so a re-upload as .pdf does not leave the old .png behind claiming to
be current. Capped at 25MB - a certificate is a document, not a disk image.
Office formats are allowed because purchase records genuinely arrive as .msg
and .xlsx, not only as PDFs.
Adding an application meant typing an image FILENAME and trusting someone had
dropped the file into the frontend's own directory by hand, and typing an
install path from memory. Both are uploads now, following the model-image trio
that models and part photos already use.
The two differ deliberately. The image is public, because application tiles
render before anything is authenticated. The installer is not: it is licensed
vendor software, an open URL would publish it to anything that can reach the
site, and it is always sent as an attachment rather than rendered.
Installers are capped at 500MB and the size is measured by seeking the stream
rather than trusting Content-Length, which a chunked upload does not send and a
client can understate. Anything larger belongs on the share, and the error says
so rather than just refusing.
Files are chosen before a new application exists, so they are held and uploaded
once there is an id to attach them to. A failed upload leaves the saved record
alone and reports, rather than losing what saved fine.
Removing an installer only clears installpath when it pointed at the upload - a
share path was typed by a person and is not ours to wipe. The detail page reads
both shapes, since entries from the classic site hold a bare filename that is
still served from /images/applications/.
The card track was 28rem, which fits four across a full-width page but only
three once the sidebar takes its 250px - and 1920 with the sidebar is the
common case here, so the board showed three. The track is 22rem now, with an
explicit four-column cap above 96rem: left to auto-fit alone a wide screen
reaches five, and a fifth column only makes the cards narrower until the rows
they hold start truncating again.
Measured at 1366, 1600, 1920 and 2560: three, four, four, four.
The floor-plan preview drops from 500x385 to 390x300. At the old size it
covered the row it was launched from, which is the row you are trying to read.
routes.gen.js spreads `.toplevel` for every plugin uniformly, but only a
few own full-screen routes, so Rollup warned on eleven of them every
build. The `|| []` guard was always doing its job - the warning was noise,
and noise in a build log is where a real warning goes to hide.
A place for the small utilities a technician reaches for at a bench. The
plugin owns no API and no tables: every tool runs entirely in the browser,
so an air-gapped site gets them for free and a bad network cannot break
them. Adding the next tool is a view, a route, and one entry in tools.js.
First tool is a barcode/QR generator. Content is typed text, a URL, or a
CSV (content,label,copies - quoted fields and an optional header both
handled), so a batch of a few hundred is one paste. Label stock is
adjustable in inches with five presets, and the code renders to an SVG
data URI rather than a PNG: a bitmap gets downscaled to label size and
smears the module edges a scanner reads, where SVG rasterizes at the
printer's resolution with hard edges.
It also carries the dot-grid rule that is easy to get wrong by eye. A
thermal head cannot render a fraction of a dot, so a code sized off the
grid gets uneven modules; pick a DPI and the page says what the current
size lands on and what to use instead. The quiet zone is blank label
rather than white baked into the code, so it can be tuned - and it applies
to CODE128 too, which needs clear space at each end and was letting bars
run into the caption.
Tech Tools is the first bundled plugin that owns no schema, which two
guards did not model: it belongs in the universal installer profile, and
upgrade-all reports it 'no-migrations' where every plugin was assumed to
report 'ok'. The migration test now asserts that status explicitly for
schema-less plugins, so a table-owning plugin whose chain went missing
still fails.
The report exists to answer "what needs replacing and where do I get it",
and it was answering neither. The part numbers were already in the
lowsupplies payload and simply never rendered; a chip per part now shows
them, with capacity tier and page yield on hover, since a model can list
several tiers for one colour.
Asset # and Location columns are gone. Location is replaced by the
floor-plan preview the asset pages already use, hung off the printer name
via its mapx/mapy.
The IP is now the site's FQDN (printer_hostname_template, built from the
IP exactly as PrinterForm does) and links to the printer's own web page in
a new tab - the report is a worklist, and losing your place in it to visit
one printer means finding your row again. The raw IP stays on hover.
Cartridge names were ellipsised inside a fixed 120px column, hiding the
one thing being reordered. The supplies cell is a grid with a max-content
name column, so names show in full and still line up across a printer's
rows.
CSV and emailed exports follow the screen, with one row per part number so
the result is a copy-pasteable order list.
"All assets 704" sat beside "all assets in use 737", and both were correct
about different populations. The totals summed five specific asset types and
subtracted dual-bay secondaries; the status counts took every asset row of any
type with no collapse, so USB devices and hidden secondary bays inflated one
side of a comparison the layout invites. Status is now counted over exactly the
same assets the totals describe.
Warranty rows fell back to asset.name when the covered asset had no hostname,
and an asset's name is usually the MACHINE's descriptive name - which is how a
column meant to identify a PC ended up showing a machine. Hostname, else the
asset number, never the name. The machine number loses its label too: the row
is hostname, machine, state, and "machine 3015" spends a word on what position
already conveys.
Printer names now carry the floor-plan preview on hover, the same
LocationMapTooltip the printer's own page uses - a location name tells you the
room, the map tells you where to walk. Declared as map.maphover on the card, so
any card with coordinates gets it; a row without them shows a plain link rather
than being dropped.
Cards are four across rather than five. At five columns a row holding a
hostname, a machine number and a state truncates on exactly the rows that
matter. auto-fit, so two cards fill the width instead of leaving empty tracks.
Not covered by a test: the count fix. I started one and it was interrupted, and
I have not gone back for it - the assertion worth having is that in-use can
never exceed the total.
The shopfloor board lost every photo and started showing lowercase SSOs where
names belong, and nothing in the system could say why. Both resolvers returned
None on any problem and three separate bare excepts threw the cause away, so an
unreachable HR host, a rotated credential, a renamed column and a genuinely
unknown SSO all produced identical output and no log line. That is not a bug in
the resolution so much as a hole where the diagnosis should be.
GET /api/employees/resolve/<sso> returns the same answer the board gets, plus
which source produced it (directory, hrdirectory, useaccount), which mode the
directory is in, and the exception text when a source failed. It is the
difference between "the board is broken" and "the HR host refused the
connection".
The two bare excepts in the shared resolvers now log rather than pass. The
external-directory branch is the one that fails on a live site; it was the one
saying nothing.
No behaviour change to the board itself - it still falls back exactly as
before. What changed is that the fallback is now visible.
PRINTER CARD is now supplies at or below 5 percent, a new
printers_dashboardpercent setting. The report and the card want different
scopes: the report lists anything the thresholds call low, which is right for
planning an order, while the dashboard is asking what to walk out and change
today - and a cartridge at 18 percent is not that. Lowest first.
PRINTER LOCATION WAS ALWAYS EMPTY. The lookup went through db.session.get on
locationid and produced nothing even where a location is set; the printers list
has always read it through the asset relationship, so the card does too now.
WARRANTY ROWS are identified the way the floor identifies them: the PC's
hostname and the MACHINE it drives, reusing the same lookup behind the warranty
page's machine column so the board and the report cannot disagree about which
bay a PC belongs to. No dates - expired or expiring is the whole decision when
scanning a board, and the exact day belongs on the report you order from.
NOTIFICATIONS are stacked: the type in full on one line, the message beneath,
trimmed to 100 characters with the rest on hover. Inline, the type was
truncated to make room for prose that was then truncated anyway, and neither
read. A tooltip is omitted when the text was not trimmed, because one repeating
what is already on screen is noise.
Two general additions: layout: 'stacked' on a card, and map.detailtooltip.
The printer card was a name followed by a comma-joined string of cartridges and
a location - the widest row on the board, and the one running past the card
edge.
Now: the printer name links to its page and reveals its location on hover, and
each depleted cartridge is its own chip showing "Black 4%", revealing the part
number to order on hover. The percentage says something is wrong; the part
number says what to do about it, which today means opening the printer's page
to find out. Every capacity tier is listed, as the report has always done.
Two additions to the card contract, both general: 'chips' maps a row key to a
list of {text, title, level}, and 'titletooltip' puts context on the row title.
Nothing load-bearing goes in a tooltip - hover is not discoverable and does not
exist on touch - so a chip always states the fact and only explains it on hover.
Chips are bordered rather than filled: a row of solid red pills reads as an
emergency even when a cartridge is merely low.
Also repaired a self-inflicted mess. A string-slice edit used a marker that
appears EARLIER in the file, so the slice was empty and two helpers were
injected at line 1, above the module docstring. Removed; the file parses and
the helpers live beside the route they serve.
Four fixes, all from looking at the real board.
"and N more" now links to a page showing them all. Telling someone 35 more PCs
are silent and leaving them to find the list is worse than not saying it. Each
card names its own destination and a test checks it against the routes that
actually exist - a viewall pointing at a route nobody wrote is the same rot the
endpoint check already guards, just failing in the browser instead of the API.
PRINTER ROWS ESCAPED THE CARD. A flex child will not shrink below its content
width unless told to, so text-overflow never engaged and a row carrying three
cartridge readings plus a location simply ran past the border. min-width:0 on
the row parts is what enables the ellipsis; meta shrinks first because it
matters least, and the card clips as a backstop.
THE STAT TILES WERE INCOHERENT. Two counted asset TYPES, two counted asset
STATUSES, and nothing said which - with the status one labelled "Active", which
reads as "not deleted" but meant status = In Use across every type. Each tile
now counts one thing and its label says so.
PCs GONE SILENT IS NARROWER, and better for it. A PC that never reported at all
is usually a hand-made or imported record rather than a bay that broke, and a
PC that is not In Use is silent ON PURPOSE - that is the status doing its job.
Both were burying the real signal: a machine that was working, is not now, and
nobody has marked as anything else.
lastseenat already recorded it and the API already returned it; nothing
displayed it, so from the UI a healthy machine still looked abandoned - one
revision from last spring and no sign anything had looked at it since.
The history page gains a Last verified column beside Captured, and the asset
panel a Verified field. Only the CURRENT revision carries one: an older
revision was superseded, so saying it was verified today would be false - what
was verified is the configuration the PC holds now.
A current revision with no check yet says "not yet checked" rather than showing
a blank or borrowing the captured date. That state is real and temporary: the
column is new, so every chain reports it until its PC next posts.
Three faults, visible only once the board ran against production data.
BACKUPS SAID THE WHOLE FLEET HAD STOPPED. The lastseenat backfill was wrong. It
seeded from collectedat, reasoning that the last change was the last provable
moment - but an unchanged config writes no revision, so a machine whose settings
last changed nine months ago got a nine-month-old lastseenat and was instantly
reported as a dead backup. Every chain lit up at once, which is worse than no
card: it says the site is broken when it is fine.
The honest value is NULL. Before the column existed nothing recorded when a
config was last confirmed, and inventing a date does not change that. Migration
0003 clears the backfill, and staleness now IGNORES a NULL chain rather than
substituting timestamps that mean something else. A chain becomes measurable the
first time its PC posts, which for NTLARS is within a day.
TONER READ "None%". The supply dict has no 'percent' key - it is 'remaining'.
Supply names are also shortened, because "Black Toner Level 4%" spends three
words saying what the card already says.
THE CARDS READ AS WALLS OF TEXT. Rows wrapped into paragraphs and a card with
forty PCs pushed everything below it off the screen. Now: at most five rows with
"and N more", one line per row that truncates rather than wraps, meta pushed
right and dropped first since it matters least, and severity reduced to a small
dot beside an uppercase label instead of a coloured card - six severity-painted
cards read as a crisis, which is how a board stops being read.
Worth recording that none of this could fail in a test. Every one needed real
data on a real fleet.
Three plugins still declared widgets naming Vue components nobody wrote.
Converting them honestly meant three different answers, not one.
notifications gets a real card: the active notifications themselves, not a
count. "4 active" tells an admin nothing; knowing WHICH message the shop is
looking at is the point, and it is how a stale one gets noticed and taken down.
machines gets machines out of service - anything not In Use, excluding
Inventory, because a spare on a shelf is stock rather than a problem. Someone is
supposed to be chasing each of those and today they are visible only to whoever
thinks to filter the list by status.
network gets NOTHING, and its declaration is deleted rather than converted.
Network devices carry no live status - no polling, no reachability check,
nothing that can be wrong - so the only possible card is a count of how many
exist, which is precisely the always-true number this dashboard exists to get
away from. A comment records that, so the next person does not re-add it. If
reachability is ever collected, that is the card.
Also adds a contract test over every declared card: no component names, a valid
renderer and severity, and - the one that matters - the endpoint must be a REAL
route. A declaration pointing at a route nobody wrote is exactly how the old
widgets rotted unnoticed for months, and now it fails the build instead.
Wave one complete. Three cards, no new data and no migrations.
Printer supplies reuses the existing low-supplies query and its five-minute
cache; a Zabbix round-trip per printer on every dashboard load would make this
the slowest page in the app. One row per printer listing every depleted
cartridge, criticals first - a row per cartridge would report one printer three
times and read as three problems, and showing only the worst class would hide a
low cartridge behind a critical one on the same machine when whoever walks out
there wants to carry both.
While there: the low-supplies REPORT itself was including healthy cartridges. A
printer with one empty black and three full colour ones listed all four, so the
reader had to find the problem inside the row. It now lists only what needs
replacing, and the test that asserted the old behaviour now asserts the new.
Expiring warranties keeps already-expired entries on the list rather than
dropping them the day they lapse, which is how they get missed. Horizon is
warranty_expiringdays, default 90, because that suits a site budgeting
quarterly and nobody else.
Mis-numbered bays promotes check-shared-machines out of a CLI command nobody
will remember to run - it found seven bays that had been wrong for weeks. It
reports only numbers with NO child assets, so part markers legitimately sharing
an operation stay silent: that distinction is the whole card, and without it it
would list correct data beside faults and be ignored.
Printers also loses its dead component-named widget; notifications, network and
machines still have theirs.
The stale-backup card could not be built as designed, and the reason is more
important than the card. Dedup means an unchanged configuration writes no
revision, so collectedat moves only on a CHANGE. A machine stable for six
months has a six-month-old newest revision and is perfectly healthy. Keying a
staleness card on revision age would have flagged most of the fleet - exactly
the noise that makes a board worth ignoring.
Underneath that: ShopDB could not distinguish those cases at all. On a no-op
the server returned "unchanged" and wrote nothing, so "we checked yesterday and
it matched" was discarded. That fact is the one thing a backup system must be
able to prove, and the only record of it was a line in a log file on the PC.
lastseenat records the check rather than the change. Touched on every matching
post including the no-op; set on creation, since a new revision has by
definition just been seen; backfilled from collectedat or createdat so existing
rows start from the last moment the config can be PROVEN current, rather than
from now - claiming a check that never happened would be worse than silence.
The card keys on it, one row per CHAIN rather than per asset: a machine with
two part markers can have one still reporting while the other stopped, and a
per-asset view would report the machine as fine. It stays deliberately silent
about assets never backed up, because whether one SHOULD be is a question only
the manifest can answer, and guessing would list a hundred healthy machines.
The rule lives in services/staleness.py rather than the route, so it is
testable without an auth layer in the way - the same split retention.py uses.
Threshold is backups_staledays, default 3, and 0 disables the card.
Second wave-one card. GET /api/computers/dashboard/quiet lists two populations
and deliberately does not merge them into one count. A PC that reported and
went quiet is probably off, moved or broken. A PC that has NEVER reported is
worse: not enrolled, or enrolled against the wrong pc-type, so nothing enforces
anything on it and no backup of it exists. That one hides indefinitely because
nothing about it fails loudly - the same shape as the bay that carried a wrong
machine number for weeks.
Never-reported sorts above the merely quiet, then longest silence first: the
order someone should work down the list, not the order rows left the table. A
soft-deleted PC is excluded - a decommissioned machine is silent on purpose,
and listing it would train people to ignore the card, which is the failure this
whole board exists to avoid.
The window is computers_quietreporthours, default 24, because every site will
disagree with any number picked here (ADR-015). A malformed value falls back
rather than failing the card.
This also replaces the computers plugin's old widget declaration, which named a
component nobody ever wrote. Four such declarations remain and will convert as
their cards arrive.
Two fixes to the renderer found while wiring this up. Meta specs now support a
trailing unit, so a row reads 'quiet for 3 days' rather than 'quiet for 3'. And
the card styles hardcoded hex colours against the frontend standard, including
a var(--card-bg) that DOES NOT EXIST - the variable is --bg-card - so the
fallback would have painted every card white and broken dark mode entirely.
Now --bg-card, --border, --danger, --warning, --primary and --link throughout.
The frontend now calls /api/dashboard/widgets. It never had, which is why five
plugins have been declaring widgets into a void for months, pointing at
components nobody ever wrote.
Core owns three generic renderers - exceptions, metric, list - and a plugin
declares data, a shape and a link template. The mapping logic lives in a plain
module beside the component, the same split as pluginAssetPanels.js, so it is
unit tested without mounting anything: 16 tests covering row mapping, empty
handling, ordering and gating.
The behaviours worth naming, because each is a decision rather than an
implementation detail:
Cards fetch INDEPENDENTLY and a failure becomes null. One hung endpoint - a
Zabbix call, a plugin mid-upgrade - cannot blank the board. A card whose fetch
failed HIDES rather than drawing empty, because "nothing wrong" and "I could
not tell" must not look the same.
Empty cards disappear by default. A card reporting nothing every day teaches
people to stop reading the page, which is precisely how a fleet log reached
3,234 lines with 17 that mattered. A card opts into a one-line presence only
when its absence is itself news.
Severity outranks position, so an info card can never sit above a failure.
Permission filtering happens BEFORE fetching: no point firing a request that
would only 403, and the dashboard must not become a way around RBAC.
An unknown render mode is skipped, so a plugin built against a newer core
degrades instead of leaving a hole.
A row whose link substitution is missing keeps the row and drops the link -
a PC shopdb does not know still reports its failure, and that is the bay most
likely to be misconfigured.
Cards sit ABOVE the totals: what needs a person first, context second. The
existing stat cards are untouched for now.
Wave one of the dashboard proposal, built as a vertical slice so the contract
is proven by something real before the other five cards follow.
GET /api/geenforce/dashboard/failures lists entries that FAILED on their PC's
most recent enforcement cycle. Per ENTRY, not per report: "three PCs failed" is
a number, while "Install OpenText failed with exit 1603 on WJSF1234" is
something a person can act on. Only current reports count, so a failure that
has since been fixed clears itself instead of needing dismissing. Hostnames
resolve to computerids in one query so each row links to the PC, and a PC
shopdb does not know still appears - the failure is real even when the
inventory is behind, and that is the bay most likely to be misconfigured.
The data has been there all along. The only way to see any of it was to open
one PC's report modal, one PC at a time.
The widget declaration is the contract change. The old shape named a Vue
component per widget, which cannot survive a lean build where a plugin's
component may never be staged into the bundle - which is exactly why five
plugins declare widgets pointing at components nobody ever wrote. This declares
data, a generic renderer, a permission and a link template, the way ADR-010
already does for asset panels. A test asserts no 'component' key, so the old
shape cannot creep back.
empty: hide is part of the contract, not decoration. A card reporting "nothing
wrong" daily teaches people to stop reading the page, which is how a fleet log
reached 3,234 lines with 17 that mattered.
Frontend rendering comes next; the endpoint and declaration stand alone and
change nothing that exists.
The dashboard reports totals that are true every day and actionable on none of
them. Meanwhile a bay returned 500 to every collector report for a day and a
half, eleven markers' backups overwrote each other for weeks, and GE-Enforce
records failures, stale manifests and repeated self-heals that surface nowhere.
Every one of those was found by someone going to look.
Two things found while writing this shaped it. The plugin mechanism already
exists - get_dashboard_widgets, an endpoint that aggregates across enabled
plugins and isolates failures, and five plugins declaring widgets. And nothing
consumes it: the frontend never calls the endpoint and not one of the declared
components was ever written. So this finishes a half-built feature rather than
starting one.
It also proposes fixing the contract while it is still unused. Naming a
component per widget does not survive a lean build, where a plugin's component
may not be staged into the bundle at all. ADR-010 already solved this for asset
panels - declare data and a render mode, let a generic core component draw it -
and the same three renderers cover every card listed.
The rest is principles that came out of this week: empty cards shrink, because a
card that says "nothing wrong" daily trains people to stop reading, which is how
a log reached 3,234 lines with 17 that mattered; every row links to the thing;
cards declare a permission so the dashboard cannot become a way around RBAC;
thresholds are settings with neutral defaults.
Wave one is six cards whose data is already reliable. Wave two is the valuable
one: diffing what a manifest says a PC should have against what it reports
having, which nothing answers today and which is what GE-Enforce exists to
guarantee.
Product version to 0.9.0, frontend in lock-step, Unreleased notes moved into a
dated section per ADR-007. The plugin contract stays at 0.16.0: it moved after
v0.8.1 and is already recorded in this release's notes, and nothing since
touched the contract surface.
A minor rather than a patch: collector behaviour changed in ways an integrator
must know about. A reported machine number no longer becomes the PC's asset
number, it builds a controls link instead; a second PC claiming a machine is
treated as a claim rather than a handover; and a backup revision chain is now
per source PC rather than per asset.
The check-shared-machines example carried two actual prod hostnames. docs/ is
excluded from the code bundle and goes to the GitHub wiki, so an example is the
wrong place for live fleet identifiers. Replaced with placeholders.
The MECHANISM was already a documented platform contract - ADR-001 defines
partof as composition, makes controls propagate through it, and walks it first
for map-position inheritance. The part-marker work used that rail rather than
inventing one.
What was undocumented is the PATTERN built on it: several devices answering to
one identifier, each becoming its own asset filed under a parent. It existed
only as a collector behaviour for part markers plus a docstring in the device
map, so nothing told anyone how to apply it to another device type, or when not
to.
ASSET-COMPOSITION.md covers when to reach for it and when the shared identifier
is a numbering fault instead, what propagation buys, how to declare a device
type through the map or a per-site setting, what a backup kind must do to
follow the device rather than the parent, how to find the next case with
check-shared-machines, and why the parent is not disposable once devices hang
off it - deactivating it breaks filing, and a hard delete cascades through
backuprevisions.
A PC that drives a device which is its own asset had been implemented twice.
METROLOGY_TOOL_MAP covered CMM, Keyence, Genspect and wax-trace, minting a
measuring_tool. A separate path keyed on one hardcoded pc-type minted a Part
Marker machine and filed it under its operation. Both create a device, link the
PC with controls, and archive that link when the PC is re-imaged: one mechanism
with different nouns, written out twice because the second case arrived later.
That is the same trap as the site literals in ADR-015 - a pattern implemented
per instance rather than declared - and it has a known next occurrence. Part
markers already share operation numbers, and any site with two marking lasers
or two wax-trace units on one number needs identical treatment.
One SUBORDINATE_DEVICE_MAP now declares asset type, type name, naming suffix,
whether the device files partof the operation, and the relationship label. The
labels are unchanged per case on purpose: those values are in the production
database and only rows carrying them are archived by a collector push. A site
overrides or adds an entry through subordinatedevice_<pctype> settings, per
ADR-015, so the next case needs no code. A malformed override falls back to the
default rather than failing the push, because a bad setting must not stop a bay
reporting its inventory.
metrology_tool_for stays as a shim over the same map: filters.py and the older
tests read it, and unifying must not change what it returns. A test pins that.
Also adds flask relationships check-shared-machines, which finds the next 0615
rather than waiting for someone to notice duplicate backups. Several devices
legitimately sharing a number and two PCs mis-numbered at imaging look the same
from outside; the difference is whether child assets exist, so that is what it
reports. Read-only.
Controller and share credentials are currently edited as cleartext into a
script on the SFLD share, one value for the whole fleet, re-applied by
GE-Enforce every cycle. Rotating means editing that file and hoping - nothing
reports which bays picked the new value up - and sites need per-bay variation,
which a manifest scope cannot express because scopes are per PC TYPE. A scope
per bay is a spreadsheet, not a model.
ShopDB therefore owns the definitions and the targeting, and a PC receives a
decided answer rather than rules to evaluate. Targeting uses the axes
GE-Enforce already filters on plus those the asset model knows: hostname
(including -like globs), machine, model, controller OS, PC type with alias
expansion, GE-Enforce profile, and a selector over the stored DNC projection -
"every machine whose DataHost is that share" is answerable from data already
held, and it also answers which machines need a credential at all.
Overlapping matches are guaranteed rather than exceptional, so precedence is
explicit: an integer priority, then scope specificity, then rule id for
stability. The resolver must be able to explain which rule won and why, or
nobody will trust it. It reuses plugins/geenforce/filters.py; there are already
two implementations of this matching logic kept honest by a parity harness, and
a third would drift silently.
Targeting stays separate from detection, mirroring PCTypes versus
DetectionMethod: who gets a credential is a server question, whether it is
already applied is a local one, and conflating them rewrites the secret every
cycle instead of healing drift.
Stored Fernet-encrypted with the key OUTSIDE the database, write-only through
the API and masked on read, fetched with a dedicated credentials.fetch scope so
a leaked collector key does not yield controller passwords, and every fetch
audited.
PROPOSED, not accepted: it makes ShopDB a credential store, and the key becomes
a single point of loss - restore the database without it and every credential
is unrecoverable, which docs/BACKUP-RESTORE.md must state in the same change
that implements this. Registry-only provisioning is recommended as a first step
regardless, since it removes the cleartext from the share immediately and the
client helper is identical either way.
Backup-NtlarsSettings reads backups_intervalhours from /api/settings/public,
because it runs before it holds any credential, and its Get-IntervalHours falls
back to 24 on any failure. The plugin never declared the key public, so the
endpoint did not return it, the fallback fired on every PC, and the setting
looked configurable in the UI while changing nothing. The fleet log shows the
symptom plainly: "Throttled: last attempt under 24h ago", every cycle,
regardless of what the setting said.
Declared public. A collection cadence is not a secret. backups_shareroot stays
private - it is internal topology - and the test asserts both directions so a
later edit cannot quietly widen it.
Same defect as the 3D parts kiosk label prefix already in this changelog: a
logged-out reader against an allowlist its key was not on. Worth noticing that
the pattern has now bitten twice.
ShopDB is a product several sites adopt, developed at one site against that
site's live fleet, so every feature arrives carrying West Jefferson's
hostnames, networks and share paths. A scan of plugins/ and shopdb/ returns 19
site literals, of which about 8 are a shipped default or a hardcode rather than
a comment: a kiosk URL baked into a PowerShell payload a sister site cannot
override, a WJ UNC path as the backup share default, and device.geaerospace.net
shipped as a SETTING DEFAULT - inherited silently by anyone who adopts.
The mechanisms already existed; the rule did not, so each value landed wherever
was convenient. The rule: a setting with a NEUTRAL default first, a
site-namespaced directory second, seed data third. Blank must be a working
state. Fleet-wide vocabulary such as the gea-shopfloor-* imaging types is not
site data and is out of scope - it already has a per-site override.
Enforced by an addition to check-naming-and-style.sh, which CI already runs, in
REPORT-ONLY mode. Making a 19-item backlog a hard gate the day it lands blocks
every unrelated commit until someone clears it; SITE_LITERALS_ENFORCE=1 turns
it into a gate once the listed hardcodes are done. The ADR carries that backlog,
including that the display FQDN domain is defined three times in three files.
COLLECTOR-INTEGRATION.md told integrators that machinenumber maps to
Asset.assetnumber and that a real value updates it on an existing PC. That is
precisely the behaviour that returned 500 to every bay reporting a machine
ShopDB already knew, and it is the document another site reads to build its
reporter.
Rewritten to say what the field is for: it identifies the machine, the PC keeps
its hostname, and the number builds a controls link. Adds a section covering
the claim-versus-handover rule and its quiet window, the alert gate defaulting
off, and how a part-marker PC gets its own marker asset under the operation.
Backups collected before markers became assets were filed against the operation
number the PC reported. The collector now files a marker PC's backup against
its marker, but only for backups collected since; everything already stored
stayed where it was, so the new marker assets read as empty while the operation
holds a mixed history.
The attribution is exact rather than a guess. A revision records the PC it was
read from, and the collector records which marker each PC drives, so the move
is that mapping applied. Revisions with no source hostname stay put, since
nothing says which marker they came from.
Moving a revision can duplicate one the collector has already written against
the marker, so the script collapses those as part of the move, keeping the
earliest so the date a config was first seen survives. Dry run by default.
A PC first seen before the machine-number fix was created with the machine
number as its OWN asset number, and the fix deliberately does not overwrite an
existing PC's asset number. So resolving the reported number can return the
reporting PC itself, and the marker was then filed partof its own PC - which
reads, on the machine page, as the PC being the operation.
The machine-link path already guarded this case; the marker path did not. It
now refuses and says why, naming the repair: rename the PC asset to its
hostname, or create the operation asset.
Several Telesis markers serve one operation number - 0613, 0615 and WJPRT each
have more than one - so treating the operation as the marker collapsed separate
devices into a single record. Their configurations differ by COM port, so in
the backup history they overwrote each other, and no question about an
individual marker could be asked at all: how many there are, which port one is
on, which one failed.
There is one marker per PC, which makes the PC the marker's identity, so the
collector can mint the marker the same way it already mints a CMM or a Keyence
unit for a metrology PC. A marker PC now gets a Part Marker machine asset, the
PC controls it, and the marker is partof the operation whose number the PC
reports. An operation holds any number of markers.
A marker PC therefore does not claim the operation directly. controls
propagates through partof, which reference-data already seeds, so control of
the operation still follows from controlling its marker - without two markers
contesting a link only one of them can hold.
Backups from a marker PC resolve to the marker rather than the operation, and
fall back to the machine number whenever the marker cannot be resolved: no
hostname on the payload, a lean build without the computers or machines plugin,
or a marker PC that has not reported to the computers collector yet. Filing
under the operation is the old behaviour and beats rejecting a backup.
Moving a marker to another operation archives the old membership rather than
deleting it, so where a marker used to live stays answerable.
Dedup compared a posted config against the latest revision for the ASSET,
which is only correct when a machine number means one PC. Several PCs share one
here: the part markers on 0613, 0615 and WJPRT are separate devices, differing
by COM port, filed under one machine number. Each marker's post therefore
differed from whichever marker had posted last, nothing ever deduped, and the
table grew by one row per PC per collection cycle.
A chain is now (asset, kind, source hostname). An unchanged config is a no-op
again, and each PC keeps its own history against the machine. NULL sources -
rows written before the column was populated, and hand-loaded ones - form their
own chain via IS NULL; `column == None` never matches in SQL, so without that
those rows would have re-posted forever.
Two consumers assumed the old key and are fixed with it. Retention pruned per
asset, so a busy marker's revisions could evict a quiet marker's only backup;
it now prunes each chain separately, protecting the newest and oldest of each.
The revision diff compared against the previous revision on the machine, which
across two markers reported one device's COM port as a change on the other; it
now compares within the source's own chain.
scripts/collapse_duplicate_backup_revisions.py cleans up what the old rule
wrote. It removes only a revision whose hash repeats the one before it in the
same chain - rows the fixed code would never have written - and keeps every
genuine change, every chain's newest and oldest, and every source. Dry run by
default. Its --report mode explains what grew each chain, which separates a
legitimately shared machine number from two PCs wrongly carrying the same one,
and from a value inside the config that changes on its own.
Treating "another PC is linked to this machine" as proof of replacement was
wrong. A PC imaged for machine 3010 carries that number from the bench, before
it has replaced anything, and several PCs sharing one machine number is a
normal state at this site: the part markers do it. Both PCs then reported on
their own schedules, each report moved the link and raised an alert, and the
pair traded the machine back and forth for as long as both were alive.
The PC holding a machine now keeps it while it is still alive. Alive means it
has reported within MACHINE_CLAIM_QUIET_HOURS and its asset is still In Use. A
challenger is recorded as a dormant link instead, which doubles as the marker
saying the claim has already been announced, so a PC sitting on a bench does
not alert on every collector cycle.
The handover still happens on its own once the old PC has been quiet for a day,
which is what a PC pulled off a machine does. Moving the old PC off In Use -
Retired, Inventory, In Repair - hands the machine over on the next report,
which gives IT a one-step way to force a swap the moment it happens rather than
waiting out the window. A day is long enough that a PC switched off overnight,
or one behind a network outage, never loses its bay to a spare.
Alerts for both cases are gated on a new computers_machinelink_alerts setting
and ship OFF. Several part markers legitimately share a machine number here, so
the alerts would fire on correct data. Links, warnings in the collector
response, and archived history are unaffected; only the sending is gated.
Also: the alert goes through send_alert rather than resolving recipients by
hand, which had missed the SMTP_ALERT_RECIPIENTS environment fallback, so a
site configuring SMTP by environment would have got the webhook and no email.
A bay reporting machinenumber 3015 got a 500 from the collector every five
minutes since it was imaged, and would have forever: the reported number was
written to the PC's own assets.assetnumber, which is uniquely indexed and
already held by machine 3015, so the insert failed with "Duplicate entry '3015'
for key 'ix_assets_assetnumber'" and the entire report was discarded. Operating
system, boot time, applications, printers and access protocols never landed.
Every retry did the same thing, so there was no path out of it.
A new PC now takes its hostname as its asset number, which is what the data
already shows: of 289 computers none has a numeric asset number and 214 use
their hostname. An existing PC's asset number is left alone; overwriting it
renamed the PC onto the machine's identifier, changing how that PC is
identified everywhere else.
The machine number instead does what it was collected for. It resolves the
machine and links the PC to it with a 'controls' relationship carrying a
collector:machine origin label, the same discipline the printer and
measuring-tool links use, so a link made by hand is never archived by a
collector push. Reporting a different machine archives this PC's previous link;
a machine ShopDB does not know is reported as a warning rather than invented.
When another PC was already linked to that machine it has been replaced. The
old link is archived rather than deleted, so which PC ran a machine in a given
month remains answerable, and an alert goes out by email and webhook. The
retired PC's status is deliberately not changed: the collector cannot tell
whether it was shelved, sent for repair or re-imaged for another bay, and
guessing would overwrite what a person set.
A line of the form "Name=" with nothing after the sign is not strictly legal,
but it occurs in real exports - the part marker's WJPRT.reg has KRelay1 like
this. The parser raised on it, which failed the whole file, which meant that
machine could never be backed up at all. Read it as an empty string so the
value name is still preserved.
The machine map preview was a bespoke popover. LocationMapTooltip already
existed and is what the machine detail page uses, so the warranty tables now
show the same thing.
That is what was asked for - the preview zooms, as it does on the machine page -
and it comes with behaviour the bespoke one did not have: scroll-wheel zoom, and
staying open while the pointer is on the tooltip itself so it can actually be
read and panned.
It also solves the clipping properly. The previous commit reached for
position: fixed with a hand-rolled flip because .table-container's overflow-x
clipped an absolute child; LocationMapTooltip teleports to body, which avoids
the clipping context altogether rather than escaping it. All of that
positioning code, and the blueprint rendering, is deleted - 64 lines from 140.
An unplaced machine keeps its chip and now says why in the native tooltip
rather than showing an empty panel: 17 of the 142 linked machines have no map
position, and those rows still need their number.
The bottom row's preview was cut off behind the pagination controls.
.table-container sets overflow-x: auto, and ANY non-visible overflow makes an
element a clipping context, so the absolutely-positioned preview was cropped at
the container's edge. The last row had nowhere to open into.
The preview is now positioned fixed from the chip's bounding rect, which
escapes the clip entirely, and flips ABOVE the chip when there is not room
below. It also clamps horizontally, since the column sits well to the right on
a wide table and the panel is nearly 300px.
Height is measured from the rendered element rather than assumed: it depends on
the blueprint's aspect ratio, which differs per site. That means one tick where
the element exists but is unplaced, so it starts hidden and is revealed once
positioned - otherwise it flashed in the corner of the screen.
Also sets white-space: normal. The table sets nowrap for its cells, which the
preview inherited and which ran the location line off the panel.
The Add User dialog was see-through over its own overlay in dark mode. Both it
and the audit-log detail dialog set background: var(--bg-card), which is
translucent in dark mode - frontend/CLAUDE.md already names this trap and says
a modal panel must use var(--bg-card-solid). The shared Modal.vue and the
.modal rule in style.css were already correct; these two hand-rolled panels
were not.
Audited every modal-ish panel in the app while here. The remaining
var(--bg-card) uses are correct and deliberate: .relationships-section and
.permission-category are cards, and the audit-log .filters rules are form
controls - translucency is the intended look on all of them.
One more was worth changing: the warranty asset-search dropdown floats above
the form INSIDE a modal, so a translucent panel let the fields show through the
suggestions. That one is now solid too.
A shopfloor PC is bought, warranted and replaced as a PC, but it is FOUND by
the machine it drives - nobody walks the floor looking for an asset number. The
warranty tables listed the covered asset and left the reader to work out where
that is.
Both tables gain a Machine # column. The payload resolves it by walking the
asset relationship graph in BOTH directions: the canonical edge is
PC --controls--> machine, but a dual-bay pair carries controls on both bays and
hand-made links are not reliably oriented.
Hovering the chip shows the floor map with the machine marked, so the row
answers "where do I go" without opening anything. The blueprint follows the
viewer's theme and the marker is placed from mapx/mapy as a percentage of the
configured map dimensions, since the preview is a few hundred pixels wide
rather than the full plan. It renders only while hovered, so a long table does
not build a blueprint per row.
A machine with no map position still gets its chip and says so, rather than
being dropped: against real data 142 PCs resolve to a machine and 125 of those
are placed, so 17 rows would otherwise have silently lost their number.
The chip is deliberately not a link. /machines/:id is keyed by machineid, not
assetid, and resolving one to the other here would make the warranty plugin
import the machines plugin (ADR-014). Worth noting separately: the existing
assetLink() in these tables already sends machine-type assets to
/machines/<assetid>, which is that same mismatch and predates this change.
Reported from a real display: Edge closed on the desktop but still listed in
Task Manager, and the watchdog never relaunched it.
That is this watchdog's own bug. It asked only whether a --kiosk process
existed. After an Edge update the window can be gone while the process lingers,
so the check said "kiosk is up" and returned - every cycle, forever. Matching on
the command line was chosen to stop a stray renderer masking a dead kiosk; it
does not help when the orphan is the parent.
A kiosk now counts as up only when a --kiosk process still owns a visible
window (MainWindowHandle). Windowless ones are killed BEFORE relaunching:
leaving them would satisfy the next cycle's check again, and a second browser
would fight the first for the display.
Also refuses to run as SYSTEM. MainWindowHandle is session-scoped, so a SYSTEM
caller reads 0 for a perfectly healthy kiosk and would kill and relaunch it on
every cycle. The task uses an interactive Users principal so this cannot
normally happen; the guard makes a mis-registered task fail loudly instead of
thrashing a display in a hallway.
Verified on Windows this time by letting the SCHEDULED TASK do the work rather
than invoking the script by hand - which is what the first version was missing:
- task fired unattended (rc=0) and launched the kiosk into session 1
- the next cycle saw the healthy kiosk and did nothing, no relaunch loop
- a windowless --kiosk process was killed and replaced
- run as SYSTEM, it refused and the healthy kiosk survived
Every time on the board was wrong, from two faults stacked.
The shopfloor feed serialised starttime/endtime with a bare isoformat(). Those
columns are stored NAIVE but hold UTC, so an untagged string is read by the
browser as LOCAL and every card shifted by the tz offset. The model's to_dict
already learned this - its _utc_iso helper documents the exact symptom, a 14:34
notification showing 18:34 - but the feed had not, so the feed now uses it too.
The dashboard then formatted with toLocaleString, i.e. the VIEWER's zone. A
board hangs on a wall in the plant: it has to read plant time whatever the
machine driving it is set to, and a kiosk with a wrong system timezone would
otherwise show wrong times to the floor with nothing to reveal it. It now
loads site_timezone and formats through formatInZone, the wall clock included -
a header disagreeing with the cards beneath it is worse than either being
wrong alone.
startsWhen was worse still: it decided TODAY/TOMORROW from browser-local
calendar days, so the wording itself could differ between the board and a
remote admin looking at the same card. That arithmetic now runs on the site's
calendar day.
Separately, the type chip carried a margin-bottom while the state chip beside
it did not. .chip-row centres each item's MARGIN box, so that margin lifted the
type chip about 4px and left "Starts Thu, Aug 13 8:00 PM" looking low. The row
already provides the spacing, so the chip's own margin is gone.
The accessprotocols / computeraccess tables replaced the old isvnc/iswinrm
booleans and the PC page already badges what a machine exposes, but nothing
kept them current: the 574 rows in place all came from the legacy migration and
have not moved since. The collector schema had no field for them.
Adds 'accessprotocols', a list of catalog names, synced with the same
discipline as the printer links. A reported protocol is activated; a
catalogued one the PC did NOT report is deactivated rather than deleted, so a
manual portoverride survives a service being briefly down. An unknown name
warns and is skipped: the catalog is admin-managed, and a typo on one bay must
not invent a protocol for the whole site.
Presence of the key is what drives the sync. A payload without it leaves every
existing row untouched, which is what protects the migrated rows from a
collector that does not report protocols yet.
Six tests cover recording, case-insensitive matching, deactivation on removal,
the omitted-key no-op, an explicit empty list meaning "exposes nothing", and
that an unknown name never creates a protocol.
Kiosks were updating, closing Edge, and never coming back - the display stayed
dead until the next logon or reboot.
The kiosk is launched by an all-users Startup shortcut, which runs ONCE at
logon, and nothing supervised the browser afterwards. RelaunchNotification=2
was meant to cover the update case and does not: that policy drives Edge's own
update-restart, which depends on session restore to return to where it was.
Kiosk mode restores no session and has no UI to show the notification in, so
Edge honours the close and never the relaunch. The same gap swallowed crashes
and anyone closing the window.
Adds a scope entry that registers a scheduled task in the INTERACTIVE session -
SYSTEM cannot launch a visible browser, which is why the dispatcher writes a
shortcut rather than calling Start-Process. The task relaunches from that same
shortcut, so the target URL keeps one source of truth: retarget a subtype in
DISPLAY_TYPE_TARGETS and the watchdog follows unchanged.
Two details that matter. It matches on the COMMAND LINE, not the image name:
Edge runs a crowd of msedge.exe children and only the parent carries --kiosk,
so testing "is msedge running" would let a stray renderer mask a dead kiosk
forever - verified against a real kiosk PC showing 7 processes and 1 match. And
it avoids -RepetitionDuration [TimeSpan]::MaxValue, which serialises out of
range and is rejected, exactly as the kiosk installer documents.
A launch debounce stops a display that fails to start from spawning a browser
every cycle, the log is size-bounded because this runs forever on a PC nobody
watches, and it does nothing at all when no kiosk shortcut is present so it
cannot put Edge on a PC that never asked for one.
Verified on Windows: registers with the right principal and triggers, relaunches
when the kiosk is gone, debounces an immediate re-run, and is idempotent across
cycles (the staged script compare is trimmed - Set-Content adds a trailing
newline the here-string lacks, so an untrimmed compare rewrote it every cycle).
The link had a custom hover tooltip that took the hidden events with
dayEvents.slice(3) - correct while dayMaxEvents was a fixed 3, wrong the moment
that cap started varying with the row height. On a short row showing one chip,
'+4 more' sliced from index 3 and listed the wrong events; on a day with three
events it sliced to nothing, hit the empty-list guard and rendered no tooltip at
all. A link with nothing behind it.
FullCalendar's own popover replaces it rather than the arithmetic being fixed:
no index to drift out of step with the cap, a header and a close button, every
event for that day listed, and it works on a touch screen - which a hover
tooltip never did on a kiosk. Clicking an event in the popover still opens the
detail modal.
That takes the hover handlers, the container mouseenter/mouseleave delegation,
the tooltip markup and styles, and the by-date index that existed only to feed
them: 108 lines out, 25 in. The popover is themed through the CSS variables,
since FullCalendar ships it light.
An audit of every scrollable element on the page, rather than the document
alone, found what the previous two attempts kept missing: FullCalendar's
day-grid scroller sits 16px short of its own content at every window size.
It is phantom. The table inside measures exactly the scroller's height, the
last week sits flush with the bottom, and the horizontal axis is clean - there
is nothing to scroll to, but the browser paints a bar for the gap. Shrinking
the calendar to close it does not work either: the gap stays 16px however small
the grid gets, so the previous pass had started shrinking the month for
nothing. That correction is reverted to page overflow only.
The grid is sized to fit its six rows, so the scroller has nothing to reveal;
hiding it is safe and is what the earlier arithmetic was failing to buy.
Audited with 60 events across 12 days at 1920x1080, 1600x900, 1366x768,
1280x720 and 1920x1200: no scrollable element left in the calendar, six weeks
visible, last week fully in view, badges shown with a '+N more' where a short
row cannot hold them all.
The nav sidebar still scrolls on a short window - 100vh of viewport against
about 1384px of menu - but that is every page, not this one.
The previous attempt at this was verified against an empty month, which is why
it measured clean and still scrolled: put events in the month and the fixed
height clipped the grid, leaving the last week behind FullCalendar's own
scroller. The media-query floors it added are gone.
Two things were guessed and are now measured. The height came from
calc(100vh - 230px), a stand-in for chrome that is not 230px tall; it is taken
from the container's real top at mount and on resize, then corrected against
whatever the page still overflows by. And dayMaxEvents was a fixed 3, which
does not fit an 80px row - hence the overflow. It is `true` now, so
FullCalendar shows as many chips as the row genuinely holds and rolls the rest
into a '+N more'; that is the only setting that cannot outgrow the box. The
chips and the day-number strip are tighter, so more fit before that happens.
With 44 events across 11 days, at 1920x1080, 1600x900, 1366x768 and 1280x720:
all six weeks visible, no page scrollbar anywhere. A full-height display shows
every chip; a laptop shows one and a link.
The floors that keep the month grid from squashing - 620px on the calendar and
110px per day cell - add up to more than calc(100vh - 230px) yields on a 768 or
720 tall window. The floors won, the page overflowed, and a month that used to
fit needed scrolling to see.
They relax below 900px tall: 78px cells, which still show the day number and
three events, so the grid stays readable rather than collapsing to the strips
the floors were guarding against. Taller windows keep the roomy cells.
Measured at 1920x1080, 1600x900, 1366x768, 1280x720 and 1080x1920 portrait:
six rows and no page scrollbar in all five.
Two of the board's colours were still hardcoded: carousel headings were gold
and grid headings blue, left over from when carousel meant Recognition and grid
meant Recertification. A type wearing another type's colour is worse than no
colour at all, and it was the whole of the "why did my category turn blue"
puzzle - it had turned from one style's hardcoded colour to another's.
Headings now take a colour from the types beneath them. Where a row holds
several types, the one with the most cards wins, ties going to whichever sorts
first on the board. Live cards decide it; the upcoming set only gets a say when
nothing on the row has started yet, so a heading always describes what is
happening now. That covers the standard rows too - the old getSectionClass
painted the list red off a legacy 'danger' keyword.
Upcoming cards say so plainly: STARTS TOMORROW 2:06 PM, or the weekday and date
beyond that, on an amber chip, with a dashed border and a hollow carousel pip.
Shape as well as colour, so it reads from across the floor and does not depend
on telling amber from white.
Cards and tiles also carry the type's colour as a bar down the left edge, the
way the legacy board marked them; a 2px border reads as chrome at distance.
Rows ran in display-style order and then alphabetically, so what led the screen
was an accident of styling and the alphabet - a new type called Awareness
landed above Recertification for no better reason than the letter A.
Each type now carries a board position, lowest first, set on the Notification
Types page. The migration seeds Recognition at 10 and Recertification at 20 and
leaves everything else at 100, so an existing board keeps the order sites
already expect. Steps of ten leave room to slot a row in without renumbering
the rest.
A row shared by several types sits wherever its earliest-ordered type puts it,
so a category moves as a unit.
Backup timestamps read wrong because of two faults stacked, which is why it
looked like a single offset.
The API serialised naive ISO ("2026-08-07T12:00:00"), with nothing saying the
value was UTC. JavaScript's new Date() parses that as BROWSER-LOCAL, so every
timestamp shifted by the viewer's offset before any timezone formatting ran.
Every datetime this plugin stores is naive UTC, so the wire format now carries
a trailing Z.
The history view then formatted with toLocaleString(), i.e. the viewer's zone,
ignoring the site_timezone setting entirely. It now loads that setting and
formats through the shared formatInZone helper, matching NotificationsList.
The panel list label is built server-side with strftime, so a client cannot
correct it afterwards. It now converts to the site zone using the same Setting
lookup the notifications plugin uses - without that it showed UTC, four hours
out at West Jefferson.
Tests cover the wire format and that 16:30Z renders as 12:30 in
America/New_York.
DNC Info becomes a single tabbed card - General, eFocas, Serial, NTSHR and
MARK - instead of a flat wall of every value. On 3204 that is 11 rows visible
rather than 22, and on 0600 eleven rather than 27, which also stops the card
unbalancing the detail page's two-column layout.
Everything DNC now lives on that one card, so the Part Marker panel is gone:
its settings are the MARK tab. The partmarker KIND is untouched and still
stores, dedupes and serves revisions - they are listed on the backup history
page - it simply contributes no card of its own, which on 145 of 147 machines
would have been an empty box.
Downloads are named for the machine: 3204.reg, and 3204-wow6432node.reg for
the dialect that imports outside NTLARS. The view had been rebuilding the name
from sourcefilename and producing 3204.reg-wow6432node.reg, so the revision now
carries assetnumber and both ends agree. That needed a viewonly relationship to
Asset - no backref, so the core asset side gains no dependency on this plugin.
The history page was hardcoded to light colours (#e0e0e0, #f4f9ff, #666) and
rendered as a white table on a dark page. It now uses the palette variables
throughout, per frontend/CLAUDE.md. The current-revision tint is a color-mix
against --primary so it reads in both themes rather than a baked light blue
that disappears on dark, and the diff columns are headed as well as red/green,
since colour alone does not survive a colourblind reader.
A panel with many fields was unusable. The detail page is a two-column
multicol (.content-column is display:contents, so the left/right wrappers are
dissolved and cards flow into one balanced flow), and a card cannot be split
across a column break. One 30-row panel therefore became a single unbreakable
block that dragged its column far past the other.
The tabs mode renders sections one at a time, so such a panel stays the height
of its largest section. Payload is {sections: [{label, fields}]}; sections that
end up with no fields are dropped rather than left as empty tabs. The active
tab is marked by an underline AND weight, not colour alone.
keyvalue rows now sit in two aligned columns instead of space-between: a ragged
right edge is hard to scan, and one long value (a UNC path) otherwise pushed
its label away from every other row.
Four specs cover the new mode, including that it drives panel visibility.
Every machine was getting a Part Marker Configuration panel, and only two of
the 147 known machines are part markers. A kind applies to an asset TYPE, but
whether a given asset ever carries that kind of backup is a property of the
individual machine, so type alone cannot decide what to show.
The generic renderer already handled this: a list panel is visible when it has
rows OR declares empty text. Declaring emptytext on both kinds defeated it and
forced them to render everywhere. emptytext now defaults to None on the base
class, neither bundled kind sets one, and the panel builder OMITS the key
rather than emitting null - a present-but-null 'empty' would still have kept
the panel on screen.
The DNC Info card carried empty text that could never be displayed, since
keyvalue visibility is decided purely on field count. Removed rather than left
to mislead.
A lathe now shows DNC Info and NTLARS history; 0600 and 0614 additionally show
Part Marker once something collects for them; a machine with no NTLARS data
shows no backup panels at all. The emptytext mechanism stays available for a
kind that genuinely wants to say "expected here, nothing yet".
site-profile-universal.json is what the released Windows installer is built
from, so a bundled plugin missing from it is invisible to the install wizard.
Worse, `flask plugin prune-schema` drops the tables owned by plugins the site
did not install (ADR-014), so backuprevisions would have been dropped at
provisioning on every new site - a table that shipped in the build, removed
because the profile never named it.
The same omission explains why `flask plugin upgrade-all` skipped backups:
upgrade_all_plugins iterates the REGISTRY, not the plugins directory, on
purpose - a plugin folder merely sitting on disk unadopted must not have its
DDL run as a side effect of a deploy. instance/ is gitignored, so any machine
that never ran `flask plugin install backups` has it on disk but unadopted.
Nothing in the suite caught the stale profile, so this adds two guards: every
bundled plugin carrying a manifest must appear in the universal profile, and
the profile must not name a plugin that does not exist. Verified the first one
fails with the profile as it was.
The view linked to /assets/<assetid>, which is not a route. Each asset type
has its own detail route keyed by its own id (/machines/<machineid>), and
there is no generic asset detail page to return to, so the link 404'd.
Uses router.back() instead: this page is only ever reached from an asset
panel, so history is the correct destination regardless of the asset type,
and it needs no per-type route table. Falls back to the machines list when
opened directly from a pasted URL.
The /api/assets/<id> call that fetches the heading name is unaffected - that
endpoint does exist; it was only the frontend route that did not.
Scheduled notifications with a special display style rendered nowhere at all.
styledGroups only read notifications.current, so anything upcoming fell through
to the standard Upcoming list, which drops banner/carousel/grid styles. A
scheduled banner was therefore invisible until the moment it went live. Upcoming
items now join the styled groups, tagged so they can be told apart, and each row
sorts live cards ahead of not-yet-started ones.
State chips name what a card is doing: UPCOMING with its start time, or
RESOLVED for the tail a type's grace window buys it. Pending cards also dim to
0.72 so a glance never mistakes one for live. The dim rule is scoped to
.recognition-card.active.pending on purpose: inactive carousel cards sit at
opacity 0, and an unscoped .pending outranks that, which stacks the whole
carousel back into view at once.
Rows keyed by board category can hold more than one type, and the heading then
speaks for none of them. Such rows keep a neutral heading instead of wearing
the first type's colour, and each card carries its own type chip. The chip is
named rather than only coloured, since colour alone does not survive a
colourblind reader at board distance.
Carousel rows gain one pip per card, so a reader can see how many are in the
rotation and how long until theirs returns.
Adds a kind-pluggable backups plugin. Configuration captured from a PC is
filed against the MACHINE it controls, with a revision history and download
back to the native format.
NTLARS/DNC is the first kind. Settings live in the controlling PC's registry
but describe the machine, so revisions attach to the machine's asset and carry
no foreign key to the PC: history survives a PC being replaced or deleted, and
sourcehostname records the handover.
Storage splits by kind. Parseable kinds store a dialect-neutral JSON
projection in ShopDB and re-render on download; opaque vendor formats (part
marker and similar) keep their bytes on the SFLD share with ShopDB holding
metadata and the UNC pointer.
Two .reg dialects exist in the wild: NTLARS's own Save... export omits the
WOW6432Node path segment, scripted exports include it. Parsing strips whichever
root matched, so a stored revision commits to neither and download offers both
(NTLARS Load... by default, WOW6432Node for direct reg import). Getting this
backwards is silent, so the dedup hash deliberately excludes sourcedialect and
both dialects of one config dedup to a single revision.
Dedup is load-bearing: the collector runs every GE-Enforce cycle across the
fleet, so a revision is inserted only when the content hash differs from that
asset's latest for that kind.
A freshly imaged PC opens NTLARS with a blank General tab. Recording that would
make an empty config the newest revision exactly when someone needs the last
good one, so a blank MachineNo is rejected rather than accepted as a change.
Two of the 320 known-good backups on the share already have that shape.
DNC Info card summarises the latest revision on the machine page: General
(Cnc, NcIF, HostType), eFocas, Serial, NTSHR when populated (only 18 of 147
machines), and MARK when the machine is a marker. MARK is gated on Cnc=MARKER
or the ShopDB machine type, not on the MARK key having content: MARK carries
serial defaults on 145 of 147 machines and DncPatterns reads YES on 103
including ordinary lathes, so neither identifies a marker.
The info card is owned by the kind (BackupKind.infopanel/buildinfo) and served
by a generic endpoint, so the expected successor to DNC ships its own card by
adding a class rather than changing the plugin or the panel wiring.
Also: schedule and retention settings with a prune that never drops the newest
or the oldest revision, and scripts/import_ntlars_backups.py to seed history
from the existing per-machine .reg files (144 of 147 resolve to assets).
Codec verified against all 320 real backups: round-trips clean through both
dialects. Bay-side generation verified on Windows against reg.exe export.
The carousel gold and the grid blue were hardcoded in the dashboard's CSS,
which was fine while carousel meant Recognition and grid meant Recertification.
Now that any type can take either style, a type arrived wearing somebody else's
colour: a green Kudos row came out gold, and every grid row came out
Recertification blue regardless of what was picked in the type settings.
Row headings now take the type's colour, and cards and tiles take theirs
through a --accent property, so a shared category row shows a red incident tile
beside a purple change tile. Heading text colour is computed from the
background's luma rather than listed per colour, since a site picks its own
hexes - gold needs dark text, navy needs white.
The card backgrounds were gold- and blue-tinted gradients; they are neutral
dark now so the accent is what carries the colour. The old values remain as CSS
fallbacks for a card that somehow arrives with no type.
--bg-card is deliberately translucent in dark mode (rgba(0,0,61,0.4)) so cards
glass over the page; --bg-card-solid exists for the things that must not. Two
hand-rolled modal panels used the former, leaving the notification-type and
access-protocol editors transparent over the overlay with the table legible
through them. The shared .modal in style.css already got this right.
Also writes down the page-vs-modal rule the codebase already follows, since
nothing stated it: a record with a detail page gets a routed form page, a lookup
row that only exists inside its list gets a modal over that list. Plus the modal
rules from the overlay-close fix - data entry never closes on a stray click,
confirmations may, and panels are painted solid.
The network device form had no IP field, so the one thing people look up a
switch for could not be entered. The API had accepted an ipaddress since the
legacy import work - create, update and every read already carried it - but
nothing in the UI ever sent one, which also left the device off the "Devices on
this network" list, since that matches on the IP a device does not have.
The field sits next to Hostname and round-trips through the Communication row
the platform keeps IPs in, the same way the PC and printer forms do.
Clearing it now clears the stored address. The upsert helper returned early on a
blank value despite a docstring promising it cleared - harmless while no form
could submit one, wrong the moment this field existed.
The shopfloor board grouped cards by display style alone, so every type set to
grid landed inside the Recertification row and every carousel type inside
Recognition's - under a heading naming somebody else's type. Setting Awareness
to grid put awareness messages under "Recertification Required".
Each type now gets a row of its own, titled by its own name, and rotation state
is per row: two carousel rows advance on their own indexes instead of sharing
one counter, and two grid rows page independently.
For the other direction there is notificationtypes.boardcategory. Types sharing
a category share one row under the category name, so Change, Awareness and
Incident can sit together while Recognition and Recertification keep their own.
Blank - the default - means a row of its own. The category is part of the
grouping key along with the display style, since a category cannot merge a
banner with a row of tiles.
A card that names no employee now renders its message as the tile or card,
rather than a placeholder face above a blank name, which is what a grid type
like Awareness looked like before.
The layout fingerprint that makes open kiosks reload now covers the category
and the grace window, so a re-grouped board reaches screens that are already up.
Eight tiles across left each one too narrow to read at floor distance. The row
now shows four per page in a 2x2 block, cycling on the same seven-second timer,
so a name is legible from where people actually stand.
The tiles lay the photo beside the name rather than above it. Two rows of
stacked tiles would roughly double the section's height, and the board is
scaled to fit one screen, so height spent here shrinks everything else. The
photo grows to 120px and the name to 26px in the space that buys.
The shopfloor feed kept every ended notification up for a hardcoded 30 minutes,
flagged resolved. A card with an 8:00 end time was still on the board at 8:29,
which reads as an expiry that did not work - and in the carousel, grid and
banner sections it read that way with no visual sign at all, since only the
standard cards render the resolved state.
The tail is now notificationtypes.gracewindowminutes, set per type on the
Notification Types page and defaulting to 0, so an end time means what it says.
A type whose cards are worth acknowledging after they clear - an incident, say
- opts into a tail, and only that type's cards get one.
The feed widens its query to the largest configured tail and then holds each
row to its own type's window. That keeps one portable query rather than a
per-type interval expression in SQL, and with every type at 0 it collapses to
"still showing".
Also fixes resolved serializing as null rather than false for a card with no
end time, which the and-chain produced.
The kiosk runs full-screen on a wall-mounted display, where the interface sat
against the top edge with the rest of the screen empty below it. It now
centres as one block, vertically and horizontally.
The title was the other half of it: the header was space-between, which reads
as centred only on the steps that show the Start over button. On the first
step, with no button to balance it, the title sat alone at the left edge of
the column. The title is centred and Start over is taken out of the flow so it
keeps its corner without shifting the title on the steps that have it.
Centring uses `safe center`, with plain `center` as the fallback line. On a
screen too short for the content, plain `center` overflows in both directions
and the header ends up above the scroll origin, unreachable. `safe` falls back
to top-aligned there and the page scrolls normally.
Three defects, all found on printedparts_label_prefix, all one root cause:
nothing in the framework knew that setting existed.
The parts kiosk runs logged out. An unauthenticated read of a setting is
limited to an allowlist, the key was not on it, so the kiosk got a 404 and
fell back to no prefix. An admin previewing the same page while logged in saw
the prefix, which is why it looked like it worked.
The same setting also looked like it would not save. The row did not exist on
a site that installed the plugin before the setting was added, so the first
save created it - under the placeholder category the settings API uses for
keys it does not recognise, where the plugin's settings page, which lists by
category, could no longer see it. The value was in the database the whole
time.
And the row was missing in the first place because seeding ran from
on_install / on_enable, which fire only on a state transition. Neither runs
again on an upgrade, so a setting added in a later plugin version never
reached a site that installed an earlier one. The comment claiming enable ran
every upgrade cycle was simply wrong.
A plugin now declares the settings it owns in get_settings_defaults(): key,
default, type, category, description, and whether a logged-out page may read
it. The framework seeds declared keys at install, at enable, and on every
flask plugin upgrade-all; files a first-time write under the declared
category; re-homes any row left in the placeholder category, value untouched;
and answers an anonymous read for keys marked public. Core carries no list of
any plugin's keys.
Contract 0.16.0 (additive optional hook). printedparts and printers move to
the hook and floor their core_version at 0.16.0. The dev database had two rows
in the misfiled state (printedparts_alert_email, employee_db_host); the first
repairs itself on the next upgrade pass.
Reordering meant clicking the up arrow repeatedly - moving a slide from the
bottom of a long playlist to the top was a dozen clicks and a page of
re-rendering. Rows are now draggable, with a grip so it looks it.
Applies to BOTH surfaces: the manager already switches between Lobby Display and
Shopfloor Screensaver, so one change covers the lobby TV and the EventSaver
playlist.
The drop target is shown as a line on the row being dropped against rather than
by shuffling rows under the cursor, which reads as the list fighting the drag.
The hover preview is dismissed when a drag starts, or it would sit over the list
for the whole gesture.
Drag and the up/down buttons now share reorderTo(), so both persist through the
same call and both recover the same way: a failed save reloads from the server
rather than leaving an order on screen that looks saved and is not.
dataTransfer.setData is set because Firefox starts no drag at all without it.
The thumbnails are 120x68 and cropped with object-fit: cover, so any text on a
slide is unreadable and the edges are cut off. Picking the right slide to
reorder or delete meant opening images by hand to tell them apart.
Hovering a thumbnail now shows the whole slide, bounded by the viewport rather
than the image so a 3300x2550 upload does not fill the screen, with the filename
underneath.
Fixed position rather than inside the row: the list scrolls and a
relatively-positioned parent would clip it. pointer-events: none so the preview
can never sit between the cursor and the move or delete buttons.
Somebody standing at a display who wants to hold a slide, or go back to one that
has just passed, had to wait for the whole rotation to come round again.
Left and right arrows step back and forward; PageUp and PageDown do the same, so
a presenter remote works without being told about it. The listener is on window
rather than an element because a display has nothing focused.
Stepping restarts the rotation timer instead of leaving it running. Advancing by
hand and then having it move again a second later, because the existing timer
was nearly up, reads as the display ignoring the keypress.
Ignored entirely when there is less than one slide to move to.
The viewport-relative height can resolve smaller than the grid needs on a short
window - a laptop with a docked browser, or a display in portrait - which would
have reintroduced the squashing it was meant to fix. Three floors now: the
container, the calendar root, and each day cell.
The day cell is the one that matters: six week rows cannot render shorter than
about 660px however short the window, so the grid stays readable rather than
collapsing back to strips.
The calendar was set to height 'auto', which sizes each week row to its own
content, so a month of mostly empty days collapsed into thin strips. It now
takes a viewport-relative height and expandRows shares that evenly across the
weeks, with a floor under each day cell so a short window squeezes the grid back
down rather than the rows vanishing.
New notifications now have "Show on Shopfloor Dashboard" ticked. The board is
where these are meant to be read, and starting unticked meant most were written
and then never appeared on it.
Only the default for a NEW notification. Editing an existing one still loads its
stored value, so nothing that was deliberately turned off gets flipped back on,
and the column default is left alone so an API or import caller that omits the
field keeps the behaviour it has today.
The installer on the share was stamped 0.8.0 and contained sixteen commits that
were not in the v0.8.0 tag. A build that misreports its own version is exactly
what the version check in docs/RELEASING-WINDOWS.md exists to prevent, and it
would have left two sites unable to say what they were running.
Everything in 0.8.1 landed after v0.8.0 was tagged this morning, driven by two
sites entering real data for the first time: the blank-code 500, the two-slide
display that never rotated, modals discarding a part-filled form, filters
returning an empty page, model photos that could not be saved, and the shared
equipment catalog that lets a new site start with vendors, models and printer
supply part numbers already present.
CHANGELOG gains a 0.8.1 section, and the OpenAPI document follows __version__
rather than being restated.
CustomFieldsInputs renders nothing when a site has defined no custom fields,
which is the normal state. The network form was the only one wrapping it in a
fieldset, and a fieldset draws its border whether or not anything is inside, so
an empty bordered box sat below Notes on every new network device.
Machines, PCs, printers and measuring tools all place the component bare. This
now matches them.
Operators reported losing a part-filled form by clicking slightly outside it.
Every data-entry modal closed on a backdrop click with no warning and no way
back - the worst possible response to a misplaced click, and it happens most to
someone adding their first records at a new site.
Close-on-overlay is removed from 35 modals across 30 files: anything containing
an input, textarea, select or v-model. They still close by Cancel or the X.
Confirmation dialogs keep it, because a delete prompt holds nothing to lose and
dismissing one by clicking away is the behaviour people expect. VendorsList
shows the distinction - its edit form no longer closes that way, its delete
confirmation still does.
The shared Modal component now defaults closeOnOverlay to FALSE. Every current
caller holds a form, a checkout, a stock adjustment or a map position being
picked, and not one passed the prop, so all of them had the same fault. A modal
that genuinely wants dismissing that way opts in explicitly.
Also regroups the operator console menu, which had grown to numbers 1-9 plus
three letters bolted on with no order to them. Actions are now grouped by what
they touch, keyed by their first letter, and the old numbers still work so
nobody who has used it for months is stopped by a rearrangement.
The menu also warns when the server is not fully provisioned and names the key
that fixes it, instead of reporting it as ordinary status lines that read as
normal unless you already knew what to look for. That check is cached for the
session because it shells out to flask twice and the answer does not change
while somebody reads the screen.
The installer offers the catalog as a tick-box, which covers new sites and
nobody else. A site that installed before it existed, or unticked the box and
later changed its mind, had to be talked through an RDP session and a flask
command - which is exactly the sort of thing the console exists to avoid.
`shopdb-admin.ps1 catalog`, and C on the menu. It runs the dry run FIRST and
prints what would be added, then asks before writing: somebody running this on a
site that has been live for years deserves to see what it would touch before it
touches anything. Answering anything but yes leaves it alone.
An older build without the seed-catalog command is reported as such and told to
update, rather than the failure being read as an empty catalog.
Repair is now on the menu too. It was reachable only by typing the verb, which
is little use to the operator most likely to need it.
`flask seed reference-data` wrote a dozen generic model types and no vendors or
models at all, so adopting this platform began by retyping a catalog another
site had already spent a year building. That is the largest single obstacle to
standing a new facility up.
scripts/export_catalog.py dumps the catalog from a live instance to
shopdb/data/catalog.json, and `flask seed catalog` loads it. What travels:
vendors 53, models 128, modelsupplies 146, modeltypes 35, machinetypes 21,
computertypes 10, printertypes 9, networkdevicetypes 5, locationtypes 11,
operatingsystems 14, measuringtooltypes 8, notificationtypes 3,
accessprotocols 3
The 146 printer supplies are the most useful part after the models themselves:
every toner, drum and maintenance kit with its part number, colour, capacity
tier and page yield, already matched to the right model, instead of somebody
reading them off spent cartridges.
IDEMPOTENT and ADDITIVE. Records match on a natural key - a vendor by name, a
model by vendor plus model number, a supply by model plus part number - so a
second run adds nothing, and it never updates or deletes: a site that corrected
a description or pointed a model at its own photo keeps its version.
Catalog only. No assets, locations, employees, business units or anything with a
serial number: nobody wants one plant's machines appearing at another. Vendor
contact details are excluded too, since a rep's name and number belong to
whoever holds that relationship. supportteams, printerdrivers and customfields
are site-specific and deliberately absent.
Models and supplies reference their vendor by NAME rather than id, because ids
differ between databases and an id-keyed catalog would silently attach part
numbers to the wrong printer.
The installer offers it as a tick-box on a new "Starter data" page, defaulting
to on, passing -SeedCatalog to stage 3. Offered rather than assumed: a site that
machines nothing does not want 21 machine types cluttering its dropdowns.
Verified by loading into an empty database and running twice: every group
populated on the first pass, "Catalog already present, nothing to add" on the
second.
Two things a second site ran into.
The network device form asked for the map position as two raw numbers, so
placing a device meant reading coordinates off another screen and typing them
in. Machines, PCs and printers have had a "Set Location on Map" picker all
along, and the network API already accepted mapx and mapy - only the form was
missing. Same picker, same modal.
The 3D parts kiosk hardcoded 'WJ' as the prefix shown before the number box,
with a comment inviting whoever needed something else to edit the source. That
is West Jefferson's gage-lab tag format and nobody else's, so another site's
operators were told to expect letters that are not on their labels.
It is now printedparts_label_prefix, set in Settings, defaulting to EMPTY - a
site that has not set one sees no prefix rather than inheriting another site's
convention. West Jefferson sets it to WJ once. The kiosk hides the prefix
entirely when unset and falls back to no prefix if the setting cannot be read,
because a cosmetic hint must never stop a kiosk working.
Not to be confused with printedparts_code_prefix, which mints item codes like
3DP0042 and was already configurable. That is the code we generate; this is the
tag already printed on the label.
A site added two slides to the lobby display and it never changed between them.
onMounted awaited fetchSlides, which starts the slideshow itself once it has
more than one slide, and then started it AGAIN unconditionally. Two timer chains
ran, and the second assignment to slideTimer lost the handle to the first, so
nothing could ever cancel it. Both fired about ten seconds later, milliseconds
apart, and each advanced one slide.
With exactly two slides that is 0 -> 1 -> 0 every cycle: the display looked
frozen. With three or more it advanced by two and merely skipped one, which is
why this survived so long - and why adding a third slide would have appeared to
"fix" it.
onMounted no longer starts it; fetchSlides owns that. scheduleNextSlide also
cancels any pending timer before setting a new one, so a future double-call
replaces the chain rather than leaking an untracked one.
While here: the feed has always sent a per-slide duration and the display
ignored it, hardcoding ten seconds, so a slide set to hold for a minute changed
after ten. It now uses the slide's own value, and the progress bar animates over
that same duration instead of finishing early and sitting full.
A server whose migrations or seeds never finished does not fail politely. Most
pages answer 500 and settings endpoints answer 404 for keys that were never
created, which reads as a broken application rather than an unfinished install.
One site spent a morning being debugged that way.
`shopdb-admin.ps1 repair` runs what stage 3 of the installer runs: db upgrade,
plugin upgrade-all, and the three seeds. Every step is idempotent, so running it
on a healthy server changes nothing, and each step runs independently so one
failure does not silently skip the rest.
`check` now says so before anyone has to infer it:
THIS SERVER IS NOT FULLY PROVISIONED
- seed data is missing (permissions, settings or reference data)
Most pages will answer 500 until this is fixed. Run:
shopdb-admin.ps1 repair
That needs a real test to sit on, so `flask db-utils seed-state` reports each
seed group and exits non-zero when any is missing. Verified by emptying the
settings table inside a transaction: MISSING, exit 1, rollback clean. Without it
the console check would have looked reassuring while testing nothing - an older
build with no such command reports UNKNOWN rather than healthy, for the same
reason.
A site reported "internal server error" adding a second business unit. It was
reproducible: create one with a blank code, create another with a blank code,
500.
A column that is unique and nullable accepts any number of NULLs - that is what
makes "optional but unique" work - and exactly ONE empty string. The form sent
'', so the first blank code saved and every one after it collided with it. The
field showed no asterisk because it genuinely is optional; the database just
behaved as though it were not.
This is not specific to business units. A dozen columns across core and the
plugins are unique and nullable - asset numbers, hostnames, item codes, subnet
names, gage-lab tags - and each was one blank form away from the same 500.
Fixing them an endpoint at a time would have left the next to be found by a
user, so a before_flush listener normalises blank to NULL on any unique nullable
text column. Listening on Session rather than on individual mappers covers
plugin models imported later, and avoids mapper-event semantics that differ
between SQLAlchemy versions.
A genuine duplicate is now a 409 with a readable message rather than a bare 500
with a traceback in the log: reusing a code that is taken is the caller's
mistake, not a server fault.
Verified against the development database: three business units with blank codes
all save, the blank stores as NULL, and a real duplicate code returns 409.
Two unrelated things found while looking at blank printer types.
Selecting a filter while past page one returned an empty list. The filter asked
the server for page 5 of a result set that now had one page, and the screen said
nothing matched. useListQuery already resets the page - setSearch and setExtra
both do - but the filter dropdowns bypassed it and called the loader directly.
Nine list pages now route through applyFilter, which calls setPage(1) when it
needs to and loads directly when already on page one, so the composable's URL
watcher does not also fire and fetch twice.
scripts/retype_models.py addresses why printer types cannot be derived. The
catalog types every printer model "Printer": true, and useless, since it does not
say whether the product is a laser, a plotter or a label printer. That answer is
a property of the model - every VersaLink C405 is a laser MFP - but nothing
recorded it, so nothing could derive it. Recording it on the MODEL means the
existing backfill fills every printer by exact name match, and a printer added
later inherits the right type the moment its model is chosen.
It exports the models needing a decision to CSV with a type suggested from the
model number, a person corrects the column, and applying it is a dry run unless
given --commit. A suggested type is refused unless it already exists in that
asset class's own vocabulary, which is what keeps the later name match working.
The suggestion order matters and got this wrong first time: a generic plotter
pattern matched "Zebra ZT411" and filed a label printer as a plotter. Brands now
come before generic patterns, and the review step exists precisely because a
confident wrong guess would type every asset using that model.
Verified on the development database: 24 printer models need a decision, 22 got
a sensible suggestion, applying them let all 42 printers match a printertype by
name, and the transaction rolled back cleanly.
The machine's own type is blank for the 134 machines that came from the classic
ASP database on machinetypeid=1, a LocationOnly placeholder the import refuses
to carry across as a real subtype. The catalog model knows what those machines
are, and its type is populated, so the column reads modeltypename under a
heading that says so.
Where both values exist they are identical - all 262 machines in the development
database match exactly - so nothing is lost by showing the one that is reliably
filled in.
This does not fix the underlying gap. A null machinetypeid also excludes a
machine from the map's subtype filter and drops its marker to the default
colour, and no column heading affects that. Only populating machinetypeid does,
which is what the backfill script is for.
Two fields on the same page were both labelled "Type": the asset's own, and the
catalog model's. Only one of them was vague. "Model type" already says exactly
what it is; the bare "Type" did not say whose.
So the unqualified one is the one that changes. No new vocabulary, and "Model
type" reads correctly against it:
Type -> Machine Type (machines)
Type -> PC Type (computers)
Type -> Printer Type (printers)
Type -> Device Type (network devices)
Left alone everywhere the word is not ambiguous - measuring tools, subnets,
VLANs, notifications, supply types and the manifest editor have no model type on
screen to be confused with.
This is a labelling change only. It does not address the blank type column on
machines imported from the classic ASP database, which is a data gap the
backfill script fills; renaming a column heading was never going to put values
in it.
Adding a "Model type" row next to "Type" put two rows in the Hardware section
that read identically. They come from different tables - modeltypes is the
catalog-wide list spanning every kind of asset, machinetypes is machine-only -
but the names line up in practice: all 262 machines in the development database
match exactly, which is the same fact that makes the type backfill safe.
So the row now appears only when the two disagree, which is the case worth
seeing: a model catalogued as one thing fitted to an asset recorded as another.
When they agree it says nothing and is hidden. Applied to machines, PCs,
printers and network devices, each compared against its own type table.
Correcting an earlier judgement. I said the model's type could not be used to
fill an asset's type, because modeltypes is the catalog-wide list covering every
kind of asset - it holds "Access Point", "Camera" and "Desktop PC" - and only
about two thirds of its names exist as machine types.
That is true across the whole catalog and misleading in practice. Restricted to
the models an asset class actually uses, the picture is different: all 262
machines in the development database map exactly, because the non-machine
entries are never used by machines. The blanks on the machines list are rows
whose type the database could already have supplied.
So the backfill now fills the type as well, under a rule that cannot mistype
anything: exact name match or nothing. A model type with no identically named
entry in the asset's own type table is reported with a count and left untouched,
so somebody can decide rather than have a guess written into their data. The
same shape covers computers, printers and network devices, each against its own
type table.
Verified against the development database by nulling one machine's type inside a
transaction: it was detected as fillable, the proposal read "LocationOnly" ->
"LocationOnly", the update restored exactly the original id, and the rollback
left the row unchanged.
Still a dry run unless given --commit, and a table missing the model column is
skipped, so it runs against a server whose network migration is not yet applied.
An asset that carries a model but no vendor was showing a blank the database
could already answer: the model records its vendor, and both sides reference the
same vendors table. Machines, PCs, printers and network devices now fall back to
it.
The fallback is FLAGGED, not merged silently. to_dict sets vendorfrommodel and
the detail pages render "(from model)" beside the value, because the record
itself is still empty: the edit form shows an empty vendor box, and a page
implying the vendor is stored would be lying about where it came from.
The model's type is exposed under its own name, modeltypename, and shown as a
separate "Model type" row. It is deliberately NOT used to fill in the asset's
own type. modeltypes is the catalog-wide list covering every kind of asset - it
holds "Access Point", "Camera" and "Desktop PC" alongside the machine entries -
so it is a different taxonomy from machinetypes. Only about two thirds of the
names overlap, and mapping one onto the other would mistype the remainder, with
the failure mode being a machine labelled "Desktop PC".
scripts/backfill_vendor_from_model.py writes the derived vendor down for real,
since the display fallback leaves reports that read vendorid still seeing
nothing. It is a dry run unless given --commit, fills only rows where the
asset's vendor is NULL and the model names one, and never overwrites a vendor
somebody chose. It skips a table lacking either column, so it runs against a
server whose network migration has not been applied yet.
Verified against the development database by nulling one machine's vendor inside
a transaction: it was detected as fillable, restored to exactly its original
value, and the rollback left the row untouched.
FLASK_ENV is not forced by the script. The app already reads it from .env, and
overriding it demanded a SECRET_KEY the environment had no reason to supply.
Three faults around vendor-model photos, found while looking at why an uploaded
image did not appear.
Saving a model was blocked after uploading a photo. The Image URL field was
type="url", and an upload sets it to an application path such as
/api/models/image/model-120.png. Native url validation demands an absolute URL
with a scheme, so the browser refused to submit the form with "Please enter a
URL" for a value the page had just written itself. The field is now type="text",
which is what it always needed to be: it holds either a full web address or a
path on this server. documentationurl stays type="url".
The upload button did not appear when adding a model, only when editing one.
That was deliberate - the photo is stored as model-<id>.<ext>, so it cannot be
sent before the record has an id - but it reads as a missing feature, and the
hint explaining it was easy to miss. A photo chosen while creating is now held
and uploaded as soon as the model is saved, and it is dropped if the dialog is
cancelled, so it cannot land on the next model created in the same session.
Network devices could never show a photo. NetworkDeviceDetail.vue binds its hero
image to networkdevice.imageurl, but networkdevices carried only vendorid, with
no link to a catalog model, so nothing could populate it - a feature that looked
present and could not work. Machines, PCs and printers have carried
modelnumberid since July. This adds the same column and relationship, the
to_dict branch that exposes modelname and imageurl, the field on the API, and a
Model selector on the form so the link can actually be set.
The migration is guarded the same way employees0002photo is: on a fresh database
the tables come from the SQLAlchemy models, which already declare the column, so
an unconditional add fails with "duplicate column name". The foreign key is
created only on databases that can add one by ALTER; routing it through
batch_alter_table made Alembic's column sort raise "Circular dependency
detected" on the fresh-database test.
Deploying this needs `flask db upgrade` and `flask plugin upgrade-all` on the
server, not just a file copy.
The exe is attached to a release, never committed: most forges reject files over
100 MB inside a repository while allowing release assets far larger, and a
committed binary would sit in every future clone forever.
Also records that tags must be pushed explicitly. A plain push of the branch
does not carry them, so a release had nothing to hang off.
The Windows installer has never shipped under a version: v0.7.0 was tagged
before any of it existed, so every build handed out so far stamped a server with
0.7.0. Two servers running different builds were indistinguishable, and the
installer logged each upgrade as "same version already installed" rather than
recording what changed. This cuts the release that fixes that.
0.8.0 rather than a patch: the air-gapped installer is a new capability, and
pre-1.0 semantic versioning puts that in the minor slot (ADR-007).
CHANGELOG gains a 0.8.0 section covering the twelve defects a real Windows
Server 2019 install surfaced, the move from inferring "is this a re-run of my
install?" to recording it, and the operator documentation.
deploy/site-profile-universal.json is now in the repository. Released builds
were being produced from a profile in a temporary directory, so the next release
could not have been reproduced once that file was cleaned up.
docs/RELEASING-WINDOWS.md points at the committed profile and says why.
scripts/gen_openapi.py reads __version__ out of shopdb/__init__.py instead of
restating it. Its hardcoded copy had already drifted a release behind, which is
the same mistake that once shipped an installer stamped with the wrong version.
The installer inferred that question from whatever the server happened to look
like: a MySQL service exists, the database has tables, the site exists, the venv
exists. None of those record who created them. A retry after a failed first
install was therefore taken for an upgrade of somebody else's working system,
which produced two dead ends on exactly the retry the wizard invites: stage 3
demanded a mandatory backup of a database its own failed attempt had written,
and then refused to prune tables it had created minutes earlier, because core
migration 7d05 seeds access protocols owned by the computers plugin and any
profile without that plugin hit the refusal every single time.
An install record at ProgramData\ShopDB-Flask\install-state.json answers it
instead. It is written when provisioning STARTS rather than when it finishes,
because the run that dies halfway is precisely the run whose retry needs it, and
it records what this installer created as it goes, so a crashed run no longer
leaves the next one guessing from the machine.
During unfinished first provisioning the pre-migration backup becomes advisory
and prune may force, since every row present was written by an earlier attempt
of the same install. On an established install both stay exactly as they were.
The classification is deliberately asymmetric: an install predating this record
carries a version stamp and probably real data, so it is treated as established
and keeps the mandatory backup. Guessing "first run" there would arm
prune --force against live tables.
Get-CreatedItems comma-protects its return. A zero-length array returned from a
PowerShell function unrolls to $null, and $null.Count is fatal under StrictMode
2.0 - the same fault that made bundle verification fail on every install
earlier. The harness caught it before it shipped.
Tests: deploy/windows/installer/tests/test-install-state.ps1 exercises new
servers, retries, completed installs, unrecorded-but-stamped installs, records
naming another directory, corrupt records, and persistence across a crash.
tests/test_installer_state.py runs it wherever pwsh exists and asserts the
invariants as text everywhere else. Both were confirmed to fail when the prune
gate or the comma protection is removed.
pytest.ini stops collection walking into deploy/windows/installer/bundle, which
is build output holding a complete second copy of the application. Importing
every plugin twice made SQLAlchemy refuse a redefined table and the whole suite
fail to collect, on a tree with nothing wrong in it, purely because an installer
had been built first. It surfaced only when the bundle grew from four plugins to
thirteen.
Waiting for a certificate from the organisation's own certificate authority
rather than buying one from a public CA. Every server this installer runs on is
centrally managed and already trusts that root, so an internally issued
Authenticode certificate removes the unknown-publisher warning exactly where it
matters; a public certificate would buy trust on machines this software never
reaches.
Notes the interim measure that costs nothing: publish the SHA-256 through a
channel separate from the installer, since a hash beside the file is only as
trustworthy as write access to that location.
Wording avoids naming internal infrastructure, since docs/ is published.
Eleven findings, grouped by the root cause each belongs to.
Wizard input reaching a command line unchecked (ShopDBFlask.iss). Port fields
were spliced in bare and arrive as [int] parameters, so a blank or mistyped
port shifted every argument after it; both port fields are now validated as
1-65535 digits. A path ending in a backslash, which is what a drive root looks
like, ended its argument with \" and CommandLineToArgvW read that as an escaped
quote, so paths are now quoted through a helper that doubles the trailing
backslash. A drive root is refused outright as well: uninstall deletes the
application directory recursively, so installing to D:\ would have wiped the
drive on removal. The password handoff was written with SaveStringToFile, which
writes an AnsiString, and read back as UTF-8, so a correct non-ASCII password
was reported as wrong; it now goes out as UTF-8 without a BOM.
Launching without checking the result. Plugin deregistration invoked "flask
plugin uninstall" without --yes, and the command carries a click
confirmation_option that aborts with exit 1 when nothing can answer the prompt,
so it could never once have succeeded; the bare 2>&1 under EAP Stop then turned
that into a terminating error which the catch downgraded to a warning while the
plugin directory was deleted regardless. It now passes --yes, brackets the
error preference, restores the location in a finally, and keeps the code on
disk unless deregistration actually succeeded. MarkShortcutRunAs had four
quotes where it needed three, which kept the whole command inside one Pascal
literal so LnkPath was never interpolated and no shortcut ever got the
elevation flag; its exit code is now logged too.
Comparing IIS physical paths as raw strings. IIS stores the path as typed, so
it may carry environment variables or a trailing backslash. A Test-SamePath
helper now normalises both sides. That closes a real hazard in uninstall, which
matched applications on alias alone and would remove an unrelated application
of the same name under another site, unattended, since -OnFailure never
suppresses the confirmation.
Accepting existing IIS state without reconciling it. "Site already exists" took
the site however it was, so re-running with a different port left the old
binding while CORS_ORIGINS, the firewall rule and the smoke test all used the
new one, failing a working server. It now refuses with both ports named rather
than silently re-binding, and refuses a site of that name serving a different
directory.
Preflight rows drawn past the panel. The failures loop had no cap at all and
the warnings loop capped at 6, a number unrelated to the panel, which holds
about three rows. The cap is now measured from the panel height, applies to
both loops, and the footer counts what was actually left out instead of
inferring it.
Also: a failed upgrade now says the application pool is still stopped and how
to start it, rather than only "part-configured", since stage 2 stops a pool
that was serving. It is deliberately not restarted automatically, because after
a stage 3 failure the deployed code and the schema may disagree. shopdb-admin
Restart-App starts a stopped pool or site instead of recycling, which is a
no-op on a stopped pool and then reported the application as unresponsive. A
dead Write-Log line that parsed as three arguments is gone, and a preflight
warning no longer tells the operator to add a directory to a compiled exe.
Two findings from the installer review, both of which fail silently.
Database dumps were readable by every authenticated user. A directory created
under ProgramData inherits BUILTIN\Users:(I)(OI)(CI)(RX), and a dump contains
every row including the users table and its password hashes. The installer
applied an owner-only ACL, but only in the branch that CREATED the directory,
so a directory created first by the console (shopdb-admin.ps1 backup) kept the
inherited permissions and the installer could never repair it. The ACL is now
re-applied on every run rather than only on creation, and the grants are made
inheritable with (OI)(CI) so dumps written into the directory later are covered
too. shopdb-admin.ps1 applies the same hardening for the default location, and
for an operator-named path says the dump holds password hashes rather than
silently rewriting the ACL of a directory that is theirs.
Verified on Windows: before, the directory carried BUILTIN\Users:(I)(OI)(CI)
(RX); after, only SYSTEM and Administrators, and a file created inside inherits
exactly those two. Without (OI)(CI) that file would not have been covered.
The uninstaller could not remove anything in IIS. [UninstallRun] launched a
bare "powershell.exe", and the Inno uninstaller is a 32-bit process, so WOW64
resolved it to the 32-bit PowerShell, which cannot see the IIS provider. The
site, application pool and application survived, pointing at a directory that
HAD been deleted, while Windows reported a clean uninstall. It now uses the
same Sysnative path as the [Run] entry, which was the last unshielded launch
site in the file.
Two audiences, two documents. Both were only in people's heads.
UPDATES-WINDOWS.md is for whoever runs a server: updates arrive as one
self-contained exe, an update takes two to four minutes, the site is down for
that time, .env and data and any hand-edited web.config are kept, unticking a
feature never removes it, the database is backed up and verified first, and a
downgrade is refused because migrations only go forwards. It covers both kinds
of security release, application and third-party, and explains that the
CycloneDX inventory staged on every server is what answers a published
vulnerability question. It also says plainly that the exe is not signed and the
checksum is the integrity check to rely on today.
It answers one question the existing docs did not address at all: the effect on
other sites sharing the same IIS server. The application pool is isolated and
the configuration is scoped to its own path, so other sites keep their own
handlers. What IS shared gets named rather than glossed: installing the IIS
modules and writing server-level configuration recycles application pools
across the server, which can drop requests in flight and clears in-memory
session state, though IIS is never stopped and no iisreset is issued. The two
IIS modules and the single permitted rewrite server variable are machine-wide
and stay behind on uninstall, deliberately, since another site may have come to
depend on them. The bundled database option collides on port 3306 with an
existing MySQL.
RELEASING-WINDOWS.md is for whoever builds releases: the three kinds of change
and the commands for each, why bundle-lock.json must be committed, the two
dependency traps that have each already cost a release, which generated files
must never be hand-edited, and the pre-release checks. It records the two known
gaps honestly - no code signing, and compiling still requires Windows and a
person.
UPGRADE.md and OPERATE-WINDOWS.md link to the operator document.
Stage 4 deliberately leaves an existing web.config alone, because operators put
real changes in it: extra MIME maps, a /installers location, bindings, a
proxy-specific rule. Overwriting reverts those silently.
That rule had no exception, and earlier builds of this installer wrote an
<allowedServerVariables> block which is fatal on its own: the section is Deny
by default, so IIS rejects the entire file with 500.52 before
httpPlatformHandler runs. Any server already installed would therefore keep the
broken file forever, with re-running the fixed installer powerless to help,
since the first thing stage 4 does is decline to touch it.
Strip just that element, keeping every other edit, and only when it contains
nothing besides the variable this installer adds. A block holding anything else
is somebody's deliberate change and is left alone with a warning. The previous
file is copied to web.config.before-xff-fix first.
Exercised against four inputs: the file earlier builds wrote, which is repaired
and still parses as XML with the rewrite rule intact; a block with an
operator-added variable, which is left unchanged; an empty block, which is the
$null.Count trap under Set-StrictMode 2.0 and is why the filter is wrapped in
@(); and an already-correct file, which is a no-op.
The stage 5 smoke test failure was a locked config section, but not one of the
two the installer unlocks. A diagnostic collected from the server returned:
HTTP 500.52 - URL Rewrite Module Error
Module RewriteModule, Handler httpplatformhandler
Error Code 0x80070021
Config Error: This configuration section cannot be used at this path.
Config File: \\?\C:\shopdb-flask\web.config
handlers and httpPlatform were both overrideMode Allow and locked false, so
the unlock had worked. The section at fault was a third one,
system.webServer/rewrite/allowedServerVariables, which ships
overrideModeDefault="Deny". web.config declared <allowedServerVariables>
locally for the X-Forwarded-For rule, and IIS rejects that declaration
outright, failing the entire configuration before httpPlatformHandler ran.
python was therefore never launched and C:\shopdb-flask\logs stayed empty,
which reads as a dead application or a permissions fault and is neither.
Unlocking the section would let every site on the machine declare arbitrary
server variables. The installer now adds the single variable to the
server-level allow list, checking first because a duplicate add is an error,
and web.config no longer declares it. The rewrite rule is unchanged.
Verified by applying the installer's own uncommenting to the template and
parsing the result: one rewrite element, no allowedServerVariables, the rule
still setting HTTP_X_FORWARDED_FOR from REMOTE_ADDR.
shopdb-diagnose.py checked only the two sections the installer unlocks, so it
could not have named this one; the IIS error page did. It now reports the
lock state of the rewrite sections as well.
The stage 5 smoke test failing tells us only that IIS did not return 200. The
cause is in one of four places, and finding out which has taken a round trip
per guess. This gathers all four in one pass and writes a single report.
It records what IIS actually answers on localhost, 127.0.0.1, ::1 and the
machine name, including the status code and the parsed text of the IIS error
page; the site, application, pool and module state from appcmd, plus the
override state of the two config sections httpPlatformHandler needs; the
contents of web.config and the resolved httpPlatform processPath; whether the
venv can import shopdb and call create_app; the application logs, separating a
missing log from an empty one; the ACLs the pool identity depends on; and
recent HttpPlatform, WAS and W3SVC event log entries.
Secrets never reach the report. Values are read from .env first, then scrubbed
from every section before the file is written, which covers command output and
tracebacks that might quote them. A password embedded in any connection URL is
also masked whether or not it came from .env.
Standard library only, so it runs on the bundled runtime or any system Python.
Verified end to end on a Windows VM: it correctly reported a 404 with the IIS
error code for an absent application, and that localhost resolves to ::1 first.
Two defects found in the stage 4 and stage 5 logs from a Windows Server 2019
install.
The scoped config unlock ran before the thing it unlocks existed. appcmd
resolves its location argument against applicationHost.config, but the unlock
was issued from the ACL block, ahead of New-WebApplication. On a first install
"Default Web Site/shopdb" is not there yet, so appcmd returned 80070003, "the
system cannot find the path specified", and the code fell through to unlocking
the section for the entire machine. That fallback exists for servers which
refuse the scoped form; it was instead the only path a first install could
take, so every install silently granted handler delegation server-wide. Moving
the block below site and application creation lets the scoped unlock work.
The smoke test discarded the diagnosis. Invoke-WebRequest raises on any
non-2xx, and the catch block kept nothing from the exception, so a fault IIS
had already identified by status code was reported as "site did not return
200 ... check the logs". It now records the status code and the text of the
IIS error page, and prints the tail of the HttpPlatform stdout log, which is
where a Python traceback lands. It also distinguishes a missing log from an
empty one: the first means the pool never launched python, the second that
python started and wrote nothing.
A non-200 that did not raise, such as a redirect, skipped the retry delay, so
the loop could spend all twelve attempts at once and report a timeout without
having waited.
shopdb/plugins/loader.py imports packaging.specifiers and packaging.version
at module scope, but packaging was never listed in requirements.in. It was
present in every development and CI environment as a transitive dependency of
pytest, so the full suite passed while a venv built from requirements.txt
alone could not import shopdb at all.
The Windows installer builds exactly such a venv, so stage 3 failed on a
customer server with ModuleNotFoundError: No module named 'packaging', after
the runtime and all wheels had installed successfully.
Add packaging to requirements.in, recompile the hashed lockfile, and add the
wheel to the offline wheelhouse with the matching bundle-lock entry. The
recompile also picked up newer uv formatting: inline environment markers on
cffi and greenlet and shorter "via" comments. The pinned distribution set and
every existing hash are unchanged.
tests/test_runtime_dependencies.py guards the general case by scanning
shopdb/, plugins/ and scripts/ for unconditional third-party imports and
asserting each maps to a distribution pinned in requirements.txt. Test
dependencies are the blind spot for this class of failure, since they are
present wherever the suite runs and absent wherever it does not.
Two defects from a Server 2019 run that got further than any before it - the
payload verified against the lock on a real server for the first time.
EMPTY EXIT CODE. "Python install failed (exit )" on an install that had actually
worked. Dropping -Wait to make -TimeoutSec enforceable left $p.ExitCode
unreadable: PowerShell only reliably populates it on a waited process. -Wait is
restored and the trade is now explicit - exit codes are load-bearing here, 1639
vs 1603 vs 3010 is the entire diagnosis, and a bounded wait is not worth losing
them for. -TimeoutSec is advisory: logged as an expected duration so a hang is
identifiable, not enforced. The code is also read defensively now, and an
unreadable one fails loudly rather than being taken for success.
That timeout has never worked - -Wait made the block dead code from the start -
so nothing is lost that was ever there. Trying to fix it broke something that
was working, which was the wrong trade to make silently.
STALE .env PREFERRED OVER A GOOD HANDOFF. The .dbpass fallback sat in the else
of "if .env exists", so it was consulted only when .env was absent. A
part-finished install HAS an .env, holding whatever password stage 2 last wrote;
if stage 0 has since regenerated the credential, .env is stale and .dbpass is
correct - and the installer preferred the stale one, giving "Access denied" with
the right password sitting unread on disk. The handoff is now applied before the
branch, so it covers both, and only when .env points at the local server so it
can never redirect a site whose database lives elsewhere.
Five defects from the Windows-defect review, each confirmed against the code
before changing it. Four of the five only fire on a RE-RUN - and after eight
attempts a re-run is the normal case, not an edge case, which is exactly why
they survived.
Invoke-Native, three defects in one function:
- Any non-zero exit was failure. 3010 and 1641 mean "done, reboot required",
and the VC++ redistributable returns 3010 on a server with a pending file
rename - an ordinary state on a freshly patched box. It is now an accepted
outcome for the installers that can report it, logged as a warning so the
operator knows a reboot is owed.
- -Wait blocks inside Start-Process until the child exits, so the -TimeoutSec
block below it could never run. Every timeout on every MSI was decorative.
The wait is now bounded here, followed by a parameterless WaitForExit so the
redirected output is flushed before it is read.
- The Python bootstrapper ran /quiet with no /norestart, free to reboot the
server mid-install.
Stage 0 refused to run when a MySQL service existed - including the MySQL84 it
had registered itself. Every bundled-database retry dead-ended while the wizard
promised that re-running was safe. A foreign MySQL still blocks; ours is started
if stopped, and the create-the-server block is skipped. It also no longer tries
to bootstrap through a root account whose password it set on the previous run:
with the handoff present there is nothing to do, and without it there is no safe
automatic recovery, so it says what to do instead of guessing.
Stage 4's appcmd unlock used '2>&1' under $ErrorActionPreference = 'Stop', which
turns any appcmd stderr into a terminating error - so the exit-code test and the
server-wide fallback, the whole reason the block exists, were unreachable, and
the stage aborted after Python, the venv, the schema and the ACLs had been
changed.
Stage 3 ran prune-schema and treated its refusal as a failure. Refusing is the
designed outcome when a table holds rows, signalled with SystemExit(1), so
Invoke-Native killed the stage and the reporting written to explain the refusal
was unreachable. Core migration 7d05 seeds access protocols owned by the
computers plugin, so any profile omitting computers hit this on every retry.
The preflight's MySQL 5.6 index-flag check is a warning, not a blocker. It
inspects the LOCAL MySQL, which may not be the database being installed against;
stage 3 checks the one actually chosen. Same class as the HttpPlatformHandler
blocker fixed earlier.
A review of the installer for Windows-only defect classes found seven live
issues. These two would have stopped the next attempt on any server.
DOUBLE-APPLIED GUARD. Yesterday's $null.Count fix was applied at BOTH ends:
Test-BundleLock returns ,$problems, and the call site also wrapped it in @().
The comma already hands the array back intact, so the extra @() nests it and
.Count becomes 1 regardless of how many problems there are. Every install would
have failed with "the bundle does not match bundle-lock.json (1 problem(s))" on
a byte-perfect payload. Applying the same guard at both ends was worse than
applying it at neither. Verified in a Windows VM against a real bundle: clean 0,
tampered 1, restored 0.
DOT-SOURCE SCOPE. bundle-lock.ps1 was dot-sourced INSIDE
Assert-BundleIntegrity, which loads it into that function's scope - every helper
it defines disappears when the function returns. Assert-BundleIntegrity itself
worked; the next caller, Get-WheelhousePythonTag, died with "The term
'Get-JsonProperty' is not recognized". It only fires where a venv already
exists, so greenfield was fine and every retry after a part-completed install
was not. Now loaded once at script scope, guarded so the stages that run without
a bundle still work.
Both were confirmed by running them rather than by reading: the nesting with a
three-case pwsh test, the scoping with a minimal repro.
Reported from Server 2019: "The property 'Count' cannot be found on this object"
immediately into stage 2.
Test-BundleLock returns an array of problems, and an EMPTY array means the
payload is exactly right. PowerShell unrolls a zero-element return into $null,
and under Set-StrictMode 2.0 $null.Count throws - so the branch that runs when
everything is correct was the one that could not run. Every failing bundle got
past it fine, which is why nothing caught it until the 8.3 path fix made
verification succeed for the first time on a real server.
Fixed at both ends: the call site wraps in @(), and Test-BundleLock returns
,$problems so no caller can be handed $null or a bare string depending on how
many problems there happen to be.
The other .Count uses in this file were already @()-wrapped and are unaffected.
Two defects in the stage 0 bootstrap, both surfacing as "Access denied" on a
server where the operator was holding the correct password.
CREATE USER IF NOT EXISTS is a no-op on an existing user - it does NOT change
the password. Stage 0 generates a fresh password every run and overwrites
.dbpass with it unconditionally, so any path that re-runs the bootstrap over an
existing account left the handoff holding a password the server had never been
told. ALTER USER now follows each CREATE, so the stored password and the handoff
always agree.
The user was also only created for 'localhost' and '127.0.0.1'. On current
Windows, 'localhost' resolves to the IPv6 loopback FIRST, so an operator who
types localhost rather than 127.0.0.1 arrives as '<user>'@'::1' - an account
that did not exist - and MySQL answers "Access denied" naming a host they never
typed. The ::1 account is now created and granted alongside the other two.
Note the datadir guard means the first defect could not fire on a straightforward
re-run - stage 0 refuses a non-empty data directory before reaching the
bootstrap. It was still wrong, and reachable once the directory has been cleared
by hand, which is what the failure message tells operators to do.
Asked why the password box does not pre-fill from .dbpass. It should not - but
it should not have been demanding a password either.
.dbpass is the ACL'd handoff stage 0 writes when it creates the database itself,
and stage 2 already reads it automatically when no password is supplied. The
wizard, though, required a password whenever .env was absent, without checking
for the handoff. On a server where stage 0 had completed but stage 2 had not -
which is exactly what a partly-failed install leaves - the operator was blocked
on a secret the installer already had, and sent hunting for a generated password
they were never meant to handle.
Blank is now accepted when either .env or .dbpass is present, and the sign-in
page says so when it sees a handoff.
Deliberately NOT pre-filled into the password box, for two reasons. It is the
only copy of a generated password, so round-tripping it through a UI control and
back out through a temporary password file adds exposure for no benefit - stage 2
reads the file directly. And .dbpass belongs to the BUNDLED database; on the
existing-database page the operator is pointing at someone else's server, where
a locally generated password is simply the wrong answer.
Trimming the description brought the Username box back and left Password off the
bottom. CreateInputQueryPage stacks its fields below the description and neither
scrolls nor shrinks, so a field that does not fit is drawn past the surface and
simply never appears - no error, no scrollbar. Sizing the description against a
pixel budget that varies with DPI and font scaling is guesswork, and it had now
failed twice.
Connection details (host, port, database) and sign-in (username, password) are
now two pages of three and two fields. Both fit under any reasonable
description, at any scaling, without anyone having to estimate.
The upgrade hint about leaving the password blank moves to the sign-in page,
where the password field actually is. ShouldSkipPage hides both pages for the
bundled-database option, and the stage arguments read the values from their new
homes.
Reported from the Server 2019 test: the Existing database page rendered a
truncated "Username:" label and no input boxes at all below it, so there was no
way to enter connection details.
CreateInputQueryPage lays its fields out BELOW the description text. The
description had grown to include a five-line CREATE DATABASE / CREATE USER /
GRANT block, added so a DBA could be handed the exact SQL. With five fields
underneath, the last two fell past the bottom of the page surface, where they
are simply not drawn - no error, no scrollbar, just missing controls.
The description is back to three lines. The SQL moves to
docs/INSTALL-WINDOWS.md, which is where someone would look for it anyway and
where it can be copied without being retyped from a wizard page.
Wizard page descriptions are a fixed budget: anything long enough to be worth
reading twice belongs in the guide, not on the page.
A Server 2019 install reported all 96 payload files as simultaneously missing
and unexpected, with mangled names - wheels/heels/flask.whl,
python/ython/python-3.14.6-amd64.exe, mysqlclient/lient/mysql.exe. Exactly five
characters of each directory name survived, which is the difference between
ADMINI~1 and Administrator.
Inno extracts the bundle under C:\Users\ADMINI~1\AppData\Local\Temp\..., an 8.3
SHORT path. Resolve-Path kept that short form while Get-ChildItem returned the
long one, so the root was five characters shorter than the prefix being sliced
off every FullName, and every relative key came out wrong. The payload was
correct; the comparison was not - the verifier refused a perfectly good bundle.
The root now comes from Get-Item, which goes through the same provider as
Get-ChildItem so their path forms agree, and the prefix is checked with
StartsWith before being trimmed. If the two ever disagree again this throws
instead of inventing paths.
Verified against the real failure mode rather than assumed: running the check
through C:\SHOPDB~3\bundle in a Windows VM now passes.
Nothing on Linux or in a normally-pathed Windows directory could have caught
this - the short name only appears under a profile directory long enough to need
one, which is where Setup extracts.
Second failure from the Server 2019 test. The previous fix worked - msiexec went
from exit 1639 (ERROR_INVALID_COMMAND_LINE, which is why it printed its usage
dialog) to exit 1603 (ERROR_INSTALL_FAILURE), so the command line parses now and
the MSI itself is failing.
It failed in 1.1 seconds. An MSI that dies that fast has not begun installing;
it has failed a launch condition. MySQL 8.4 requires the Visual C++
redistributable and a bare Windows Server does not ship it - the same runtime
mysql.exe and mysqldump.exe import, which was visible when their DLL
dependencies were trimmed and went unnoticed.
Stage 0 now installs VC_redist.x64.exe from the bundle before touching MySQL,
skipping it when vcruntime140.dll is already present, and fails with a sentence
naming the requirement if the redistributable is absent from the bundle
altogether. vcredist\ is an optional locked payload.
msiexec also gets /l*v now. A bare 1603 names neither the failing action nor the
reason, and it is the most common MySQL install failure - diagnosing this one
took a launch-condition inference rather than a log. The MSI log lands beside
the installer's own in ProgramData, so the next failure is readable instead of
guessed at.
Reported from a Windows Server 2019 test: a "Windows Installer" dialog listing
every msiexec /Option appeared, then the wizard reported that the bundled MySQL
database could not be installed. That dialog is msiexec's usage help - it prints
it when the command line does not parse - so the install never started.
Cause: $MysqlRoot defaulted to 'C:\Program Files\MySQL\MySQL Server 8.4', which
contains spaces. Invoke-Native wraps any argument containing whitespace in
quotes, producing "INSTALLDIR=C:\Program Files\...". msiexec takes public
properties as PROPERTY=value and expects the VALUE quoted -
INSTALLDIR="C:\Program Files\..." - so it rejected the line, printed usage, and
exited non-zero.
This file already carried the rule, next to the Python target: "Never put a
space in a path this installer controls." I broke it setting the 8.4 path.
Two fixes, because one of them alone leaves the trap in place:
- $MysqlRoot is now C:\MySQL84, space-free like C:\Python314. The MySQL client
search paths in the installer, the preflight and the operator console all look
there first, keeping backups working against the bundled server.
- Invoke-Native now quotes PROPERTY=value correctly, so passing a spaced path
explicitly no longer produces an unparseable command line.
tests/test_installer_defaults.py fails if an installer-controlled path default
ever contains a space again.
The preflight page began refusing to continue while any check was failing, which
is right for something the operator must go and fix. HttpPlatformHandler was
marked FAIL when absent - so on a server without it the wizard stopped dead,
telling the operator the server was not ready, over a module the bundle carries
and stage 4 installs a few pages later. The only way forward was to go and
install by hand the exact thing the installer was about to install.
It is now INFO: reported, not blocking, matching how URL Rewrite is already
handled. Nothing the installer SUPPLIES may block the wizard, and
tests/test_installer_defaults.py now fails if that rule is broken again.
The site-port conflict check is downgraded from FAIL to WARN for the same class
of reason: it runs before the operator reaches the Address page, so it tests the
DEFAULT port rather than the one they intend to use, and blocking refuses an
install over a conflict the very next page lets them resolve.
Genuine blockers are unchanged - no IIS, no WebAdministration, wrong Windows
edition or architecture, no disk, and the MySQL 5.6 index flags. Those the
operator really does have to fix first.
Adopting a site means getting its asset register in. The HTTP import API suits a
site with a source system and someone to script against it; a sister site with a
spreadsheet and no developer needs something else, and that is the common case.
FOREIGN KEYS TAKE NAMES. This is the whole design. A CSV row has to say where an
asset is, and the database stores locationid, an integer. Requiring the number
means importing locations, reading back the generated ids and pasting them into
the asset sheet - a workflow nobody finishes. Every foreign key here accepts
either a numeric id or the referenced row's name:
assetnumber,assettypeid,statusid,locationid
CMM-01,Measuring Tool,Active,Gage Lab
The column keeps its database name, per CONTRIBUTING.md; the value is whatever
the operator actually knows. Names resolve across files in one run, so
assets.csv can reference a location that only exists because locations.csv was
read moments earlier. A name that does not resolve is reported with its line,
column and value, not as a foreign key violation from three layers down.
Dry run is the default, and writes go into the transaction either way - the
rollback is what makes it a dry run. Skipping the writes instead made every
cross-file reference fail, which is the one thing a folder-wide check exists to
verify. Validation covers every row before anything is written, so a typo on
line 400 cannot leave 399 rows imported. Files are matched on a natural key, so
correcting a spreadsheet and re-running updates rather than duplicates.
TEMPLATES ARE GENERATED, NOT MAINTAINED. "flask csv templates" builds them from
the live schema, annotated with required/optional and which file each foreign
key refers to. The prompt for this was a hand-written template set that had
invented columns on seven of eleven tables and named a table that does not
exist, while looking entirely plausible - and described an import mechanism
(a Data Import page, a flask import-csv command) that had never existed. A test
fails the build if a generated template ever offers a column the schema lacks.
User accounts are deliberately not importable: passwords do not belong in a
spreadsheet in either direction.
Verified end to end against MySQL 5.6 - a folder dry run catching one bad
reference, the fix, the commit, and a re-run reporting updates rather than
inserts. 16 tests.
The bundled-database option could not actually be built. Stage 0 looks for
mysql\mysql-8.0.x-winx64.msi, and Oracle no longer publishes a standalone server
MSI for 8.0 - every 8.0.x returns 404. What remains for 8.0 is the MySQL
Installer bundle, which is an installer-manager: 'msiexec /i INSTALLDIR=' would
install THAT rather than a database, and stage 0 would then fail on a missing
mysqld.exe.
MySQL 8.0 also reached end of life in April 2026, so bundling it would have put
an unsupported database on every new site.
8.4 LTS still ships the standalone MSI (129MB, which is what the '125MB' note in
stage 0 was written against) and is supported into 2032. Defaults follow it:
install root MySQL Server 8.4, service MySQL84. The operator console still looks
for an 8.0 install path as a fallback, for sites already running one.
Also bundles mysqlclient\ - mysql.exe and mysqldump.exe with the two OpenSSL
DLLs they actually import, 20MB rather than the 51MB of debug and auth-plugin
libraries the archive ships. Stage 2 stages it onto the server, so a site whose
database is on ANOTHER host can still take the pre-upgrade backup that every
upgrade depends on. That was the gap the preflight had started warning about.
Bundle is now 221MB.
The GitHub backend job failed on test_there_are_docs_to_check, correctly. docs/
is stripped from the published repository - it lives in the wiki on that side -
so on the mirror the glob matched nothing and the guard fired exactly as
designed.
An absent docs/ and a glob that silently matches nothing in a tree that HAS docs
are different conditions, and the test conflated them. The module now skips when
the directory is not there at all, and the guard still fails when it is there and
empty. Verified all three ways: 7 pass here, 7 skip in a docs-less checkout, and
the guard still fails against a docs/ containing no markdown.
The gate has to ship rather than be excluded from publication, because the
published tree is where the GitHub CI that would catch a regression runs.
rsync --exclude also PROTECTS a path from --delete, so anything that reached the
publication tree before its exclude existed stayed there permanently - invisible
to the sync and surfacing only as a scrub-gate failure. That cost two rounds of
'add the exclude, still fails' on the installer bundle and again on
.pytest_cache. The generated paths are now purged before the sync, so adding an
exclude is sufficient on its own.
tests/test_docs_publishable.py assembles its search terms from fragments: a file
containing the literal strings the scrub greps for tripped that scrub on itself.
Excluding the file from publication would have removed the check from the
repository it protects.
docs/ is excluded from the code bundle and its scrub gate, because it goes to
the GitHub wiki instead - via a generator that has no gate at all. So the one
part of the repository written in prose, by people, about internal
infrastructure, was the one part nothing checked.
What was reaching a public wiki: the internal git server's URL and hostname,
.gitea workflow paths, developer home directories in the GE-Enforce cutover
reference, and a dev database root password inside a copy-pasteable command in
the import guide.
All replaced with neutral equivalents. tests/test_docs_publishable.py is now the
gate, at the source, in CI - a wiki page cannot be un-published, so catching this
after the fact is not good enough.
PROJECT-REVIEW.md also referred to internal tooling by name throughout; those
references are generalised. It remains an internal candid assessment of this
project that is nonetheless published, which is worth a separate decision.
The sync walks the working tree rather than git, so deploy/windows/installer/
bundle came through despite being gitignored - about 100MB of build output
containing a copy of the whole application tree, the wheels and the vendor
installers. Its copies of config.py and requirements.txt then tripped the scrub
gate, which is the only reason it was noticed.
Excluded along with the installer's other generated files. Note that rsync
--exclude also protects a path from --delete, so a copy already in the
publication tree has to be removed by hand once.
The '# via' annotations recorded the absolute path of the temporary file the
lockfile was compiled from, which is meaningless to anyone else and does not
belong in a published artifact. They now read 'requirements.in', which is where
these requirements actually come from.
Pins and hashes are unchanged - verified by a hash-checked dry-run install.
CLIENT IP / SPOOFABILITY. docs/geenforce-api-cutover.md claimed that removing the
IIS rewrite rule made the allowlist fail closed and that it does NOT become
spoofable. The opposite is true. IIS never sets X-Forwarded-For on its own; the
rule is the only thing that does. Remove it and IIS still forwards whatever
X-Forwarded-For the CALLER sent, waitress trusts it because it arrives from
127.0.0.1, and remote_addr becomes attacker-controlled - so a token-less caller
can fetch manifests from anywhere on the network. The document and the
_trusted_client_ip docstring now say so, waitress runs with
--trusted-proxy-count=1, and stage 5 checks the rule is actually live rather than
assuming it. The wizard question is rephrased to something an operator can verify
with their network team instead of guessing at.
NON-ASCII. The style gate only ever checked .py/.vue/.js/.ts, so documentation
accumulated em-dashes, arrows and box-drawing characters against this repo's own
convention - including in files added this week. Cleaned, and the gate now uses
INCLUDES_ALL so Markdown, JSON and YAML are covered.
PLUGIN DEFAULTS. The wizard pre-ticked measuringtools and printedparts, both of
which ship default_enabled=false, so every site taking the defaults installed and
enabled them against their manifests. Inno has no JSON parser so the list must be
hardcoded, but tests/test_installer_defaults.py now fails when it drifts.
UPGRADES. The payload copy merges, so a plugin dropped from a site's profile kept
its code forever - which defeats a lean build and leaves core's optional-import
guards succeeding for a plugin the site no longer has. Stale plugin directories
are now deregistered and removed before the copy.
add-plugin used 'plugin install', which for the five default_enabled=false
plugins left them installed but DISABLED - and printed a green success line
anyway. It now goes through apply-profile, and the success line is gated on the
exit code. Invoke-Flask records its own exit status, because $LASTEXITCODE keeps
a stale value when flask.exe is missing and no native command runs.
CHARSET. The utf8mb4 compiler hook lived inline in migrations/env.py, so it
covered the CORE chain only: plugin baselines inherited the server default, which
on a latin1 server means two charsets in one database. It is now
shopdb/utils/mysql_charset.py, imported by both, and preflight reports the
database's default charset.
BACKUP HONESTY. The dump was described as 'all of your asset data'. Uploaded
branding and floor-map images live in instance\ on disk, not in the database, so
a restore from the .sql alone comes back with no map. backup now archives
instance\ alongside it and says both are needed.
VERSIONING. AppVersion was hardcoded at 0.9.0 while the product, the frontend and
the newest tag said 0.7.0 - and 0.9.0 collides with a retired contract version.
Both builders now generate version.iss from shopdb/__init__.py.
Smaller: rollback overwrites .env before deleting it, as uninstall already did;
appcmd unlocks are scoped to this site's location rather than server-wide, with
the wide unlock as a fallback; DEVELOPMENT-SETUP says Python 3.14; the README
plugin list gains printedparts; prune-schema --force is documented as
first-provisioning-only; HTTPS is documented as not-the-default with the steps to
add it; the DBA SQL is on the wizard's database page; the features page says
unticking does not remove an installed feature; and the installer README states
that bundle-lock cannot vouch for the exe itself - that needs signing or an
out-of-band hash, neither of which is wired up.
Nine fixes from a review of the installer against its actual audience: DT leads
at sister sites who are not Windows, IIS or Python specialists and who will lean
on an AI assistant to get through it.
TRUTHFULNESS. The preflight was advisory - an operator read 'IIS is not
installed', pressed Next, answered five more pages and the install died partway
through with Python already on the box. The results page now blocks while
anything is failing, repaints on every run instead of latching after the first,
and offers 'Check again' so a fixed problem does not mean starting over. On
failure the wizard said 'Nothing was left running', which is false in every path
because the stages run with -OnFailure never: it now says the server is
part-configured, that re-running is safe, and how to remove it. The final page no
longer reads 'ShopDB-Flask is ready' after a failed install.
SECRETS. The generated MySQL root password went to Write-Host in a process the
wizard runs hidden - so nobody saw it - and stdout is forwarded into the setup
log operators are told to send to support, so it was permanently recorded for
everyone who did not need it. It now goes to an ACL'd file. Database dumps, which
contain every user password hash, landed in a ProgramData directory readable by
every user on the box; the directory is now locked at creation.
UPGRADES ON REMOTE-DATABASE SITES. mysqldump was looked for only under local
MySQL install paths, so a site whose database is on another host silently skipped
every pre-upgrade backup - after stage 2 had already stopped the pool and
replaced the tree. Find-MysqlTool now prefers a client shipped in the bundle,
stage 2 stages it onto the server, preflight reports when it is missing, and
mysqlclient\ is an optional locked payload.
UNINSTALL. A subpath install is an IIS Application, not a site; removing only the
site left the application pointing at a deleted directory, so the parent site -
at West Jefferson, the live classic ASP - served 503 on that path forever while
Add/Remove Programs reported success. Uninstall now reads MOUNT_PATH and removes
the application. The firewall rule was created as "$SiteName $SitePort" and
removed as the literal 'ShopDB-Flask 8090', which matches nothing.
DAY-2 TOOLING. Every shortcut now passes -AppRoot and -SitePort, and the console
forwards them through its own elevation and 32-bit relaunches instead of
discarding them - a non-default directory or port made it report a healthy site
as broken, from a shortcut the installer wrote. 'Open ShopDB-Flask' resolved to a
hardcoded localhost:8090 that was wrong for every subpath install; it now asks
the console, which reads the address the installer recorded, and no longer
demands administrator to open a browser.
SMOKE TEST. The parent-site port lookup filtered for an http binding and
defaulted to 80, so an https-only parent site failed a working install with a red
dialog.
DOCS AND /api/docs. The installer was invisible: nothing in docs/, README.md or
CLAUDE.md mentioned it, so a DT lead or their assistant landed on the manual IIS
runbook and hand-built the very server the installer then refuses to upgrade.
docs/INSTALL-WINDOWS.md and docs/OPERATE-WINDOWS.md are now the canonical route,
the two manual runbooks are bannered as reference-only, README and CLAUDE.md
route by target, and llms.txt tells an assistant which document to follow and to
ask for 'check -Json' before diagnosing. Both ship on the server, along with
openapi.json and llms.txt - without those the self-hosted /api/docs was broken on
every installed box, which matters most to the sites least able to debug it.
Stage 5 now checks it actually serves.
shopdb-admin.ps1 gains 'check -Json': one structured, secret-free block covering
version, publishing method, IIS state, HTTP reachability, database, Python
version, plugins and errors. That is the cheapest useful answer to 'the operator
will ask an LLM' - it works with no infrastructure, which a install-time MCP
server could not.
Stage 4 decided whether the IIS objects it was about to reconcile were its own by
testing for .installed-version. Stage 2 writes that file, and stage 2 always runs
first in a '-Stage all' install - so by the time the guard looked, the stamp it
had just written made every server look like one this installer built, including
the hand-built ones the guard exists to protect.
Stage 2 now records whether a stamp was present BEFORE it writes its own, and
stage 4 reads that observation. Running stage 4 alone still tests the file, which
is correct there: no stage 2 has run to disturb it.
Found by review, not by test - the guard has no coverage, because exercising it
needs a live IIS.
An air-gapped site cannot be scanned from anywhere else, so when a CVE lands the
only way to answer 'is that component here, and at what version' was to RDP in
and go looking. The frontend was the real blind spot: nothing recorded which
version of leaflet, dompurify, jspdf or html2canvas ends up inside the compiled
SPA.
scripts/generate_sbom.py emits CycloneDX 1.6 covering both ecosystems - every pin
in requirements.txt with the sha256 the installer enforces, and every package in
package-lock.json. Build-only npm packages are marked scope 'excluded' rather
than dropped, so 'not here' stays distinguishable from 'not looked for'.
Dependency edges are real: uv's '# via' comments give the Python graph and
package-lock gives the npm one.
Hand-rolled rather than cyclonedx-py plus cyclonedx-npm because both inputs are
already pinned and committed - this is a format translation, not a scan - and
because the build box may be a work PC with nothing but Python and Node. It is
deterministic by construction: same inputs, byte-identical output, so
regenerating does not churn.
Staged into the application tree by both builders, so it installs onto the
server with the app. shopdb-admin.ps1 verify reports it and searches it by
component name, which is the question actually being asked.
Packages appearing at several depths in package-lock (node_modules/vite and
node_modules/vitest/node_modules/vite) are merged, and a copy reachable outside
the dev tree makes the component count as shipped. Emitting both produced
duplicate bom-refs, which CycloneDX forbids and scanners reject; getting the dev
merge backwards would have hidden a shipped package from a CVE search.
Not covered by bundle-lock.json on purpose: its provenance is git, not the
third-party payload.
Two guards for a server deployed by hand, which the West Jefferson production
box is.
An existing venv is reused, which is right for a repair or an upgrade of an
install this made, and wrong when the venv belongs to a different Python. The
wheelhouse is tagged for one minor version, so pip finds no candidate for the
compiled packages and dies partway through - after Python has been installed and
the application tree replaced. The two versions are now compared up front and
the run stops with both numbers and what to do about it.
Switching deployment method removes the other method's IIS artifact. That is
correct when this installer owns both and dangerous when it does not: a wrong
-MountAlias would call Remove-WebApplication on a live mount with no prompt and
no error, and the first sign would be the site returning 404. It now refuses
unless a version stamp shows this installer made the install, or -AdoptExisting
is passed, and the refusal lists exactly what it would have removed.
The lock records what IS in the wheelhouse, not what the application NEEDS, so
an incomplete wheelhouse was locked, blessed and shipped - and only failed on an
air-gapped server.
That is not hypothetical. Assembling the wheelhouse anywhere other than Windows
silently omits colorama, a win32-only dependency of click, because pip evaluates
environment markers against the machine doing the downloading rather than the
machine being targeted. The bundle built here was short exactly that one wheel.
Both verifiers now cross-check wheels/ against the staged requirements.txt,
ignoring markers, since a requirement guarded by sys_platform == 'win32' is
precisely the one that must be present. Names are normalised to PEP 427 wheel
form, so mysql-connector-python matches mysql_connector_python.
bundle-lock.json is the first real lock: 42 files, cp314/win_amd64 - 39 wheels,
Python 3.14.6, HttpPlatformHandler 1.2 and URL Rewrite. MySQL is absent and
optional; a site choosing the bundled-database option adds it and re-locks.
The naming gate now skips the installer's build output. It contains a staged
copy of the application plus a second SPA build under dist-subpath, which
--exclude-dir=dist does not match, so a staged bundle failed the gate on
vendored minified JS nobody in this repository wrote.
IIS does not set X-Forwarded-For on its own and HttpPlatformHandler connects
from loopback, so without a rewrite rule every client reads as 127.0.0.1. The
GE-Enforce IP allowlist, the dashboard visitor-location lookup and per-host
login rate limiting all stop working, silently. The rule needed URL Rewrite,
which the installer told operators to download - from an air-gapped server.
URL Rewrite now ships in the bundle, and the wizard asks which case applies,
because the two answers are mutually exclusive. Directly exposed: install it and
set X-Forwarded-For from REMOTE_ADDR, which is what stops a client spoofing its
own. Behind a proxy: leave the rule off, since REMOTE_ADDR is the proxy and
applying it would discard the real client IP.
The rule is enabled by deleting two explicit marker lines rather than by a regex
over the surrounding comment, so editing that prose cannot silently disable it.
An existing web.config is no longer overwritten. It is the one file on a server
that legitimately carries hand-edits, and replacing it reverted them without a
word - on a server where the X-Forwarded-For rule had been enabled by hand, that
alone would have turned the GE-Enforce IP allowlist off. The installer reports
what it found instead.
pip now runs with --require-hashes and --only-binary=:all:. Hash-checking is
requested explicitly rather than inferred from the lockfile, so shipping an
unhashed requirements.txt fails loudly instead of quietly dropping the check.
shopdb-admin.ps1 gains a verify command: which bundle this server was installed
from, and whether the installed packages still match what shipped.
The .iss states its compiler floor. WizardStyle uses the built-in windows11
custom style, which needs Inno Setup 6.6.0; older compilers now fail with that
sentence rather than 'WizardStyle is invalid'.
The bundle carries ~40 wheels, a Python installer and two MSIs. All of them run
as SYSTEM on the target server, and nothing verified any of them. A missing
wheelhouse printed MISSING and the script still exited 0, so an empty bundle
compiled into a shippable installer and the failure surfaced on an air-gapped
server with no way to fix it.
bundle-lock.json now records that payload exactly - sha256 and byte size per
file - and verification is set equality: a missing file, an unexpected extra
file, or changed content all fail. Both builders check it and refuse to produce
an unverified bundle; the lock ships inside the bundle and shopdb-install.ps1
re-checks it on the server before running any of it.
This is deliberately a layer above requirements.txt hashes. pip lists every
artifact of a pinned version (cffi 2.1.0 alone has 100 hashes), so it proves a
wheel is genuine, not that it is the wheel this bundle was built and tested
with; it ignores extra files in the wheelhouse; and it covers none of the
executables.
refresh-bundle-lock.ps1 regenerates the lock but refuses to overwrite one until
the operator has seen the diff, because the commit is the review - it is the
only place a change to what runs as SYSTEM becomes visible to a human.
build-installer.ps1 is the whole build natively on Windows, so a work PC needs
no Bash. It shares the plugin closure resolver with build-site.sh.
Both builders now copy the installer scripts from the repository. They were
copied from a downloads folder, so the logic that shipped was not the logic that
was committed and the build worked on exactly one machine.
Two verifiers exist because PowerShell is the only thing guaranteed present on
the target server, while the Linux builder should not need pwsh.
tests/test_bundle_lock.py runs both against the same fixtures and fails if they
disagree.
Two fixes to the lean-site build.
The closure resolution moves out of an inline heredoc into
scripts/resolve_plugin_closure.py. The Windows builder needs the same answer,
and a PowerShell reimplementation would have been a second copy of the rules,
free to drift and produce a bundle whose plugin set did not match its profile.
The backend staging step copied all of deploy/ into the output tree. The Windows
installer stages its bundle at deploy/windows/installer/bundle, so that copy
recursed into its own destination and cp aborted with 'cannot copy a directory
into itself' - the documented build could not complete. Only
deploy/windows/web.config is read at install time, so only that is staged; the
rest of deploy/ is installer source and does not belong on an application
server.
test_plugins_only_import_contract_surface has been failing on main since
9a2d0cc: the employee name resolver imported shopdb.core.models directly.
shopdb.api already exports User (contract 0.13.0), so this is the same object
reached the way ADR-001 requires.
Both files are recompiled with --universal --generate-hashes, preserving every
pinned version. Three things change.
Hashes put pip into hash-checking mode, so a wheel whose sha256 is not listed is
refused rather than installed. The offline Windows install previously took
whatever file in the wheelhouse satisfied the version pin.
--universal means one lockfile serves Linux (dev, Docker, CI) and the Windows
wheelhouse. The Linux-only resolve had silently omitted colorama, a win32-only
dependency of click; in hash-checking mode a missing entry is a hard error, so
that omission would have broken every Windows install.
requirements-dev.txt is now compiled with -c requirements.txt, pinning shared
dependencies to the versions production runs. The two had been compiled at
different times and drifted: CI tested against alembic 1.18.5 while sites
installed 1.18.4.
Hashes pin the version and prove the artifact is one upstream published. They do
not pin WHICH artifact of that version is used, and they say nothing about extra
files in the wheelhouse - bundle-lock.json covers both.
Vite compiles the mount path into the bundle, so it cannot be chosen at install
time from a single build - a page served under /shopdb would load and then
request its assets from /assets/, and render nothing.
build-site.sh now produces both:
frontend-dist base / - the app on its own IIS site
frontend-dist-subpath base /<alias> - an IIS Application under an existing
site, e.g. http://<server-fqdn>/shopdb/
SUBPATH_ALIAS (default 'shopdb') is fixed per bundle and written into the staged
build as .alias, so the three places that must agree - the IIS application alias,
MOUNT_PATH in .env, and this compiled base - cannot drift apart. The installer
checks that marker and refuses rather than serving a page that cannot load.
The subpath build runs FIRST and is held in a temp dir: the root build has to be
last so frontend/dist is left in the state a developer expects, and the copy into
$OUT has to happen after the staging step that does rm -rf "$OUT".
Roughly 2500 lines of tested installer had been living in ~/Downloads and an
untracked folder - nothing was under version control.
It goes here rather than in a repo of its own because it depends on application
internals: the `flask plugin` verbs, site-profile.json, MOUNT_PATH, and the
plugin registry. Versioned separately it would drift out of step with the thing
it installs.
Contents: the read-only preflight, the staged installer (bundled MySQL, runtime,
schema, IIS, verify, uninstall), the operator console, the Inno Setup wizard, the
bundle builder and the artwork generator.
bundle/ and Output/ are ignored - regenerable, and ~220MB. plugins.iss is ignored
because build-installer.sh generates it from the staged payload. The artwork IS
committed so a Windows build box does not need Python and cairosvg.
Verified end to end on Windows Server 2025 against a bundled MySQL 8.0 and an
existing MySQL 5.6: fresh install, upgrade with backup and rollback, re-run
idempotency, uninstall, and both deployment methods including switching between
them. Not yet verified: a hypervisor-level air-gapped run, and any load from a
real browser (every HTTP check so far used curl, which sends no Origin header).
The step was called "Starter Data" and offered to "seed the data a new site
needs", but seed_starter inserts eight vendor rows and nothing else: no assets,
locations, departments or statuses. An operator ran it, saw every dashboard
count stay at zero, and reasonably concluded the seed was broken.
Rename the step to Reference Data, describe what is actually seeded, and say
outright that no assets are created and that an empty dashboard is expected
here. Assets arrive later via the import API.
build-site.sh staged only shopdb/, the chosen plugins/ and frontend-dist, so the
output could be imported but not run or migrated. The Windows installer had to
assemble wsgi.py, requirements.txt, migrations/ and deploy/ separately, which
meant it could assemble a payload whose plugin set did not match the profile the
tree was staged from.
Stage those runtime files, and copy the profile in as site-profile.json so the
set is self-describing: `flask plugin apply-profile` at provisioning reads the
same profile the tree was built from, so installed plugins and shipped plugin
code cannot drift.
frontend-dist keeps its name; CI reads that path (ci.yml:79).
The closing hint now spells out `prune-schema --yes --force`. ADR-014's prose
says lean provisioning "uses --force", but --force alone only permits dropping
non-empty tables; without --yes the command is a dry run that prints a preview
and exits, so following the ADR literally silently skips the prune.
A fresh install landed on the anonymous dashboard instead of prompting to create
the first admin, so an operator had no way to discover /setup.
The router now asks /api/setup/needs-admin before rendering any unauthenticated
route and redirects to /login?firstrun=1 while no user exists. The result is
cached in a composable so it costs one request per session, and the lookup fails
open (a backend that cannot answer must not lock the login screen). Racing it
against a 4s timeout keeps a slow or hung backend from blocking the first paint.
Login.vue clears the flag after creating the admin so the gate stops firing
without a reload.
REQ-D: restore waitress and tzdata to requirements.in. They existed ONLY in the
generated requirements.txt (hand-added in bf9e60e), so the next
`uv pip compile` would have silently removed the WSGI server and the IANA
timezone database from every Windows install.
REQ-E: split production and development requirements. requirements.txt was
installing pytest, pytest-cov, pytest-flask, coverage, iniconfig and pluggy onto
production servers. Verified on a real Windows Server box before this change.
CI, scripts/test-external-plugin.sh and the dev docs now use requirements-dev.txt.
REQ-F: standardise on Python 3.14. The repo declared four different versions
(Dockerfile 3.12, DEPLOY-WINDOWS-IIS 3.12, INSTALL-WINDOWS-IIS 3.13, CI 3.13,
plus README, web.config and PLUGIN-EXTERNAL-REPO). 3.14 is in active bugfix
support until ~Apr 2027 and supported to Oct 2030; 3.13 entered security-only in
Apr 2026. All four compiled dependencies publish win_amd64 wheels for 3.14
(cryptography via an abi3 wheel), verified by building an offline wheelhouse and
installing it on Windows Server 2025.
REQ-G: state MySQL 8.0 as the standard for new installs; 5.7+/5.6 remain
supported on an existing server.
Lockfiles regenerated with uv pip compile. Production deps 44 -> 38.
Each marker bound both a hover tooltip and a click popup. On the map page the
click handler routes to the detail page, so the popup opened and the
navigation discarded it in the same tick - it was never visible. Where it did
render (the map editor and the picker forms) its 'View Details' link only
served to pull the user off an unsaved form.
Keep hover as a glance and leave the click to the consumer. Removes the popup
markup, its styles, and the two now-unused detail-route helpers.
The ring only has to separate the mark from the surface; at 2px it read as
part of the mark. Markers go to 1.25, legend dots to 1.5px (they are larger,
so the same visual weight needs slightly more), and the PDF marker and swatch
strokes drop to match.
Two independent ways a restart leaves the SPA unable to navigate, both of
which look identical to a user - a click that does nothing.
1. The router awaits loadEnabledPlugins() to gate plugin routes. An app-pool
restart leaves that request hanging (IIS queues it while the worker starts)
and axios sets no timeout, so the navigation never resolves. Worse, the
promise is cached, so every later navigation awaited the same dead request
and stayed frozen long after the backend recovered. Bound the wait and fail
open on expiry, and drop the cached promise when an attempt times out or
fails so the next navigation retries. The setup-state probe in the guard
gets the same bound (it already fails open, defaulting to "complete").
2. A deploy replaces the content-hashed chunk files, so a tab open across it
asks for chunks that no longer exist and the dynamic import rejects with
nothing handling it. Reload once on a chunk-load error, via router.onError
and Vite's preloadError, guarded by a sessionStorage flag against a reload
loop and cleared on the next successful navigation.
Also stop index.html being cached: it names the hashed chunks, so a stale copy
points at files the deploy already deleted. It now revalidates while the
hashed assets under assets/ cache for a year.
Keying the ring off the fill singled out the light colors: on the map the
orange network-device marker took a black ring while its neighbours kept white
ones, and the legend dot for the same type wore a surface-colored border, so
the key did not match the markers.
Ring by the SURFACE instead - dark on the white blueprint, light on the dark
one - which is uniform within a theme and still works for every fill, since a
fill that resembles the ring is by definition far from the background. Legend
swatches take the same ring, the theme watcher redraws the markers (their ring
now depends on it), and the PDF uses the light-surface ring throughout because
it prints on white.
Markers sit on two very different surfaces - the light blueprint on white and
the dark blueprint over the near-black navy card - but the palette only ever
suited one. The grey "no subtype" default sat at 1.88:1 against white and the
orange asset-type step at 2.16:1, so both effectively disappeared on the light
blueprint. The fixed white ring made it worse: on white it added nothing.
Re-step the asset-type colors to versions of the SAME hues that clear 3:1
against both surfaces, replace the grey default with a neutral that clears
5.4:1 / 3.7:1, and derive the ring from the fill's luminance (light fill ->
dark ring, dark fill -> light ring) so every marker keeps a hard edge on
either background. The ring also rescues a washed-out color a user picks by
hand for a subtype, which no palette change can reach. The PDF export applies
the same rule to its markers and legend swatches, keeping print in parity.
Colors were chosen against a contrast/CVD validator rather than by eye. Note
that five simultaneous hues cannot all stay distinguishable under color-blind
simulation - past roughly five subtypes on screen, the legend and the hover
tooltip carry identity.
Adds computed contrast assertions so a future palette edit cannot
reintroduce a washed-out step.
exportPdf passed the raw map_blueprint_light setting value, which is a
root-relative /api path. Under /ops or /shopdb that resolves to the server
root and 404s, so the export died with "Failed to load blueprint image".
The on-screen map was unaffected because it goes through blueprintUrlFor,
which applies withBase - use that here too.
Same file, so this also carries the subtype auto-palette replacement that
goes with the marker-legibility change in the next commit.
Every asset list shows a Type column (and printers a Model, machines and
network a Vendor), but the search filters only looked at the asset number,
name, serial and hostname. Searching a type returned zero rows: 'Part Washer'
on machines, 'Standard' on PCs, 'Thermal' on printers.
Extend the search on machines, computers, printers, network devices,
measuring tools and the unified asset list to cover the type name plus the
vendor/model where the list shows them. Joins are outer joins so an asset
missing a type or vendor still matches on its own fields; the core list uses
a correlated EXISTS instead, since its type-name filter already joins
AssetType.
MapView carried its own copy of the per-type subtype-id lookup and it
never gained a Measuring Tool branch, so selecting any measuring-tool
subtype filtered out all assets. Marker coloring and the PDF export were
unaffected because both already used the shared getSubtypeId helper.
Point the filter at that shared helper and delete the duplicate copy in
ShopFloorMap too, so one definition serves filter, coloring and export.
Adds a table-driven spec covering every subtype-carrying asset type.
The old kiosk kept relaunching the dead URL from an HKLM Run value the 32-bit
Inno installer wrote - WOW64-redirected into SOFTWARE\Wow6432Node, which 64-bit
tooling (and the earlier purge) never saw. Broaden the sweep to both registry
views, every loaded user hive, Run/RunOnce/Policies-Explorer-Run, matching by
legacy name AND by any value pointing at the old URLs, plus every per-user and
common Startup folder.
The white-on-login was the old Dashboard/Lobby installer's leftover autostart
relaunching the dead old URL (404 -> white), not a network race - so the
wait-for-URL launcher solved the wrong problem. Go back to the plain direct
Edge kiosk shortcut and clean up any stale launcher file. The real fix (the
legacy HKLM Run-key + old .lnk purge) stays; it just has to be published.
At auto-login the Startup shortcut fired before the network was up, so Edge
--kiosk navigated to nothing and sat on a blank white page with no retry.
Point the shortcut at a hidden VBS launcher (wscript, no console flash) that
polls the kiosk URL until it responds (up to ~3 min) and only then launches
Edge fullscreen, so the first paint is the real page. Falls through to launch
anyway after the timeout so a display is never left dark.
Notification start/end times displayed and stored wrong by the tz offset
(a 2:34 PM entry showed 6:34 PM). Two stacked bugs: to_dict emitted stored
UTC as naive ISO (no offset) so the browser read it as local, and the form
filled the datetime-local input from toISOString() (UTC).
Fix and generalize to a configurable site timezone (multi-site):
- New setting site_timezone (default America/New_York), public, editable in
Settings > Site > Localization (common-zone dropdown).
- Backend tags datetimes UTC (_utc_iso); parse normalizes to naive UTC
(_parse_utc); daily-reset expiry uses the site zone (_next_site_time);
calendar allDay events key off the site-local day (_site_date).
- Shared frontend util datetime.js (Intl-based, DST-safe) converts between a
UTC instant and a site-zone wall clock. Notification form, list, and
calendar all render/enter in the site zone.
The MCP server README names Claude Desktop / Claude Code (it is an MCP
server for those clients), which trips the scrub gate. It is a standalone
local tool distributed via pxe-images/mcp/ + setup-mcp.cmd, not part of the
shipped product, so exclude it from the public repo like docs/ and tools/.
The old LobbyDisplay/Dashboard Inno installers planted an HKLM
...\CurrentVersion\Run value (plus a Startup .lnk). The dispatcher already
swept the stale .lnk/.url launchers but never the Run value, so a display
with our new ShopDB Kiosk.lnk still relaunched the old kiosk URL at logon
(the Run key beats the Startup shortcut). Remove the two legacy Run values
and kill any running old-URL Edge so the display self-heals to the resolved
target on the next enforce cycle.
A separate tool (not shipped in the app) that exposes a curated set of read
endpoints as MCP tools, so an LLM client can query the asset DB directly. Built
with FastMCP.from_openapi over docs/openapi.json; auth via a scoped PAT
(SHOPDB_TOKEN) or managed X-API-Key. Read-only: only GETs on the curated
allowlist become tools, all writes excluded. Runs anywhere that can reach the
API - never on the air-gapped box. Needs `pip install fastmcp` + testing in that
env (not installed in this repo's venv).
Generate docs/openapi.json (3.1, 362 operations) from the API inventory via
scripts/gen_openapi.py, and serve it with a self-hosted Redoc bundle at
/api/docs - no CDN, works on the air-gapped box. Also serve docs/llms.txt (a
concise LLM entrypoint) at /api/docs/llms.txt. New core 'docs' blueprint;
staticdocs/ excluded from the naming check (vendored minified JS).
Servers were imported as computers (a PC type) so they show under PCs, not
Network. This one-shot re-points each server's asset in place - assetid is
unchanged, so comms/relationships/map/name/location/audit all carry over; only
the extension row is swapped (computers -> networkdevices), the asset type is
flipped, and the device gets the 'Server' networkdevicetype (created if absent).
Identify servers by their computer type name (--type, default 'Server'). Dry-run
by default; --commit applies. Run on the target instance.
The /printer-installer map only reads the public install-list and downloads
the install .bat - both jwt-optional endpoints - so requiring auth was an
unnecessary gate. Drop requiresAuth; it now matches the other display/kiosk
tools (public).
The Inno printer installers hand-parsed JSON in Pascal (brittle brace-counting).
Add ?format=text to install-list (one printer per line, pipe-delimited:
printerid|windowsname|vendorname|modelnumber|hostname|ipaddress|mapx|mapy) and
to pc-default (printerid|windowsname), so the installer side is a split() with
no JSON parser. The web map keeps the default JSON.
Also resolve install-list's vendorname via the model (as the batch already does),
since the import sets the model, not the printer's direct vendorid - otherwise
the installers' HP/Xerox/Brother filter drops every prod printer.
Prod had 331 relationships, 268 computers, 204 machines, but the report came
back empty. The query only matched computer(source) -> machine(target), while
the import stores the general machinerelationships as machine(source) ->
PC(target) (only the synthetic measuring-tool links are PC -> tool). So the real
shop-floor edges never matched.
Make the query direction-agnostic (UNION of both orientations); a PC-runs-machine
report is conceptually undirected. Also drop the comtypeid=1 filter so the IP is
taken from the primary communication regardless of its type.
Test: a machine(source) -> PC(target) edge now appears in the report.
Two fixes for the printer install-batch on prod data:
1. Vendor was read only from the printer's direct vendorid, which the legacy
import never sets (it sets the model; legacy resolved vendor through the
model). Every prod printer came back vendor "unknown", so all fell into the
manual group and the universal PrinterInstaller.exe block never emitted. Now
resolve vendor via the model's vendor when the printer has no direct one, as
the classic installprinter.asp did.
2. Harden the download base URL. Behind IIS the app sees http on a loopback
port and url_root drops the /shopdb mount, giving a broken download URL when
site_base_url is unset. Fall back to https + the forwarded Host + script_root.
Test: a printer with no vendorid but an HP/Xerox model now groups universal.
Rebuilds the classic printer-installer feature: pick printers on the shopfloor
map, download a .bat that installs them.
Backend (asset_routes.py): GET /api/printers/install-batch?printerids=1,2,3
returns a .bat attachment. Groups printers the way the classic installprinter.asp
did - HP/Xerox via the universal PrinterInstaller.exe /PRINTER="a,b,c", printers
with a .exe installpath via that installer /SILENT, and anything else (no
installpath, or a .zip) listed for manual install instead of being run blindly.
Download URLs derive from the site_base_url setting + the IIS-served /installers
folder (no hardcoded host). Reuses the existing install-list query shape.
Frontend: PrinterInstallerMap.vue - full-screen Leaflet shopfloor map (reuses
mapConfig), a marker per network printer at its mapx/mapy, click to toggle-select,
sidebar with the selection + an Install button that downloads the batch. Toplevel
route /printer-installer, printersApi.installList(), and an Installer Map button
on the printers list.
Tests: install-batch grouping (universal/specific/manual) + requires-ids.
Two problems with application download/launch/doc links:
1. Stored paths like 'installers/Foo.exe' are relative, so an <a href> on
/shopdb/applications/6 resolved to /shopdb/applications/installers/Foo.exe.
New basePath.fileHref() mounts a relative path under the app base
(-> /shopdb/installers/Foo.exe) while leaving full URLs and UNC/file paths
untouched. Applied to installpath, applicationlink, and documentationpath in
the list and detail views.
2. Even the correct /shopdb/installers/Foo.exe 404s: httpPlatformHandler is
path="*", so IIS forwards it to Flask, which has no such route. Add a
web.config <location path="installers"> that clears the handler and serves
that subpath as IIS static (with .exe/.msi MIME), from a physical
APP_ROOT\installers folder.
The notes field is authored as HTML (the form says "HTML supported") but the
detail page interpolated it with {{ }}, so tags like <BR> showed as literal
text. Render via v-html through a DOMPurify sanitizer (utils/sanitizeHtml):
allow-list of formatting tags + links only, forces target=_blank
rel=noopener on links, strips scripts/handlers. Promote dompurify to a direct
dependency (was transitive via jspdf).
IIS ARR sets X-Forwarded-For to clientip:port, and the port changes every
connection. Left in, the audit log showed IP:PORT, the dashboard IP fallback
never matched a stored (portless) DashboardDefault.ipaddress, and login rate
limiting keyed per-connection instead of per-host. Add an IPv6-safe
clientip.client_ip / strip_port helper and use it in the audit log, the
dashboard resolver, and the login rate-limit key.
The KB list search matched only shortdescription + keywords, so searching a
topic (e.g. "Spotfire", the Application name) surfaced just the one article
whose title/keywords contained the word, not the others tied to it by topic.
Match the topic too via an appid IN (apps named like the term) subquery - used
instead of a join so it does not collide with the sort=topic join, and articles
with no app still match on title/keywords.
The .kb-* classes had no styles, so the KB entries rendered as bare inline
spans - shortdescription and keywords (both up to 500 chars) wrapped and mashed
into one block. Style each entry as a bordered clickable card: description as
the link title clamped to 2 lines, keywords split on whitespace into small
muted chips below.
The location option label read l.location, but the Location.to_dict() field is
locationname, so every option rendered blank - the dropdown looked empty and
"massive" (a long list of blank rows). Fixed across all five affected forms:
printers, computers, network devices, network device form, and the subnets
location filter. Other .location uses (printer-driver URL, search-result label,
report bylocation key) are legitimately different fields, left alone.
Also require a model on the printer form: asterisk + required attr, plus a JS
guard in savePrinter (the native required is skipped while the select is
disabled with no vendor picked) that points the user at the vendor first.
AuditLogs: the scoped table-layout:fixed + width:100% forced the table to fit
the settings pane, so cells ellipsis-clipped (Timestamp/User/IP fell off) rather
than scrolling. Drop it so columns size to content and the container scrolls
horizontally (global .table-container is overflow-x:auto). Only the free-form
Name/ID cell stays bounded (320px + title tooltip) so one long value cannot blow
the table width out.
EnforcementReports: the per-entry detail modal capped at 640px, too narrow for
the 5-column table. Widen to min(1000px, 92vw) and let the Message column wrap
instead of forcing horizontal scroll inside the modal.
The lobby-display and screensaver slide manager was admin-only. Add a shared
slides.manage permission so a curator can manage both surfaces without full
admin. Admins keep access via the require_permission admin bypass.
Backend:
- plugins/slides/api/routes.py: all 5 management routes require slides.manage
- plugins/slides/plugin.py: declare it via get_permissions(); nav item carries
the permission so the frontend can gate visibility
- shopdb/core/api/auth.py: login response now returns the user's permissions
(matches /me) so the frontend authStore has them on fresh login
Frontend:
- stores/auth.js: hasPermission(name) getter (admin true, else granted list)
- router/index.js: guard supports requiresPermission
- views/AppLayout.vue: hide nav items whose permission the user lacks
- plugins/slides/frontend/routes.js: slide manager gated requiresPermission
Tests: no-perm user 403, curator role with the perm 200 (+ login advertises
it), admin 200 via bypass.
Deploy: run `flask seed permissions` to create the row, then grant it to a
role in Settings > Users & Roles.
Global search did a single ilike('%CSF Roles%'), so any query with more than one
word required the exact contiguous phrase and usually returned nothing. Add
_word_match: split the query into words and AND them (OR across the searched
columns per word), so 'CSF Roles' matches a record with both words in any field,
any order. Applied across every domain (assets, applications, KB, employees
[selfhosted + external HR], notifications, hostnames, IP, custom fields,
vendor/model/type). External HR path uses a parameterized per-word LIKE.
The dispatcher now derives its FQDN (F<BIOS serial>.<domain>) and asks
/api/dashboarddefaults/display-role for its role/path, so changing a display's
type/location in Settings > Dashboard Defaults takes effect with no reimage. If
there is no serial, no server mapping, or the lookup fails, it falls back to the
local display-type.txt map (offline-safe). VM-verified both paths.
Adds GET /api/computers/display-kiosks - the displays that reported in (Kiosk
type), each with its derived FQDN (F<serial>.<domain>). The Dashboard Defaults
form gets a kiosk dropdown that fills the FQDN so admins pick a display instead
of typing an IP; IP stays an optional manual field. Table shows FQDN or IP.
'Business Unit' label -> 'Location' on this page + the settings nav.
A display's DHCP IP can change; its FQDN (F<serial>.<domain>, domain from the
display_fqdn_domain setting) is stable and the collector already reports the
serial. Add a nullable unique fqdn column (varchar191 so the index fits utf8mb4
without innodb_large_prefix), make ipaddress nullable, and require fqdn OR ip.
visitor-location + display-role resolve by FQDN first, then IP; create/update
accept fqdn. Core migration 7d31, verified up/down/idempotent on MySQL 5.6.
'Business unit' wording -> 'location' in the validation messages.
replace_scope_draft deleted old draft entries with per-object db.session.delete
but left the deleted objects in scope.entries. On a re-publish a caller
(seed_display_scope) then matched a stale deleted entry via next() and
store_inline_payload attached a payload to its dead entryid, failing the
manifestpayloads->manifestentries FK on MySQL (1452); SQLite does not enforce
it so the idempotency test passed. Clear the collection via the delete-orphan
cascade instead, and flush pending inserts before the bulk payload delete so its
autoflush cannot interleave a half-built insert. Verified publish + re-publish
x3 on MySQL 5.6.
- notifications shopfloor feed: resolve the employee name live when the stored
value is a bare SSO (WJ notifications imported as SSOs, never converted), for
both single and split-per-employee cards
- employee name resolver: after a directory miss, fall back to the shopdb User
account (firstname/lastname, keyed by SSO username) so users from other
locations still show a name
- shopfloor dashboard: employee photo falls back to the GE monogram (own asset,
independent of the site_logo setting) with a loop-guarded onerror; recognition
+ recert tiles both covered
- shopfloor dashboard: 'All Business Units' filter label -> 'All Locations'
- geenforce display dispatcher: startup sweep also matches the imaging
installers' 'GE Aerospace Dashboard/Lobby' shortcuts by name
The prior sweep only matched '--kiosk'; the imaging installers (Inno
GEAerospaceDashboardSetup / lobby) create Startup shortcuts with single-dash
'-kiosk' pointing at /shopdb/shopfloor-dashboard, so they survived. Match any
msedge/chrome Startup .lnk whose args contain -kiosk (one or two dashes) OR a
shopdb kiosk URL (tsgwp00525 / /shopdb/ / shopfloor-dashboard). Unrelated
Startup items are left untouched (VM-verified).
- Resolve-ShopdbPayloads wrote an absolute local path into the entry, and the
engine resolves it as Join-Path InstallerRoot <field>, doubling it
(C:\...\payloads\C:\...\payloads\<sha>.ps1 -> PS1 not found). Write the leaf
filename instead; the runner already sets InstallerRoot to that payloads dir.
- display dispatcher now removes leftover kiosk launchers from prior installs
(any Startup .lnk that runs Edge --kiosk, plus .url to a shopdb kiosk page),
not just its own, so two kiosks do not fight.
- allowlist auth uses remote_addr, not the spoofable first X-Forwarded-For hop
(adds _trusted_client_ip + a regression test); rate-limit path unchanged
- client psm1: fix Set-StrictMode crashes reading absent keys in Get-ShopdbConfig
(token-less mode) and Resolve-ShopdbPayloads (no-payload entries); validate
the manifest response is JSON before overwriting the last-known-good cache
- runner: pass the engine its required -InstallerRoot/-LogFile; create the log
directory so enforce logging is not silently lost on a fresh kiosk
- display scope: dispatcher writes an all-users Startup shortcut instead of
Start-Process (SYSTEM cannot show a window in session 0), resolves the base
URL from HKLM, and adds an always-on power/no-lock entry; tests updated for
the 6-entry scope
Two instances run on the box (dev /ops + prod /shopdb); each needs its own
base-path build. Build both on every --dist so prod never ships a stale
frontend. tools/ is excluded from publication, so this is dev-tooling only.
discover_share only matched 'common' and 'gea-shopfloor-*', so a display/
manifest.json on the share was silently skipped and 'flask geenforce publish
display' failed with 'No scope display/runtime'. The display scope is a
first-class HTTPS-pull target (kiosks fetch pctype=display), so accept it.
The client IP allowlist config was a tab inside the GE-Enforce section; move
it to the Settings rail via get_settings_cards (matches printedparts / zabbix /
dell). Route relocated from /geenforce/settings to /settings/geenforce; the
in-section Settings tab is removed. Card: Settings > GE-Enforce.
Get-ShopdbConfig required both BaseUrl AND ApiToken, so a token-less kiosk
(authorized by the server's IP allowlist) got a null config and never ran.
Now BaseUrl alone is a valid config; X-API-Key is sent only when a token is
present (New-ShopdbAuthHeaders), so token-authorized sites are unchanged and
vaulted-network sites need no per-PC token.
Fleet PCs on a trusted (vaulted) network can now reach the GE-Enforce client
endpoints (manifest, payload, report) without a per-PC token: the auth path
accepts a valid geenforce.fetch/report token OR a source IP in the configured
allowlist (setting geenforce_allowed_cidrs). Fail-closed; an empty allowlist
means the token stays the only path, so existing deployments are unchanged.
Rationale: the client token lives in HKLM on every kiosk, so it does not
defend against a compromised kiosk anyway - network-perimeter trust is the
same practical strength with far less provisioning + no token-rotation churn
on a DB wipe. Documented in-UI that this is perimeter trust, not per-device
identity.
- _ip_allowlisted() (ipaddress, X-Forwarded-For-aware via _client_ip)
- /geenforce/config GET/PUT extended with allowedcidrs, server-validated +
normalized (bad CIDR -> 400)
- new GE-Enforce > Settings tab (GeEnforceSettings.vue) to edit the allowlist
in admin, no SQL
- 3 regression tests (allow by IP, reject outside list, empty = token required)
The bulk /sync/dell reuse check only matched an existing warranty when its
provider was exactly 'dell'. Warranties added by hand or via import default to
provider 'manual', so re-check-all did not recognize them and created a brand
new Dell warranty for every asset - duplicating the whole set.
Broaden the reuse match to treat a warranty as Dell by any signal (provider,
matching service tag, or a 'Dell' vendor), and canonicalize the reused row to
provider 'dell' so later re-checks match by provider and never duplicate.
Air-gapped sites cannot pip install / npm ci / docker pull, so a build-at-site
compose (build: .) fails and reports 'service api is not running'. Add a
build-once-ship-image path:
- scripts/build-offline-bundle.ps1: on a connected box, build shopdb-flask +
pull mysql:8.0, docker save both into one gzipped tarball with a sha256.
- docker-compose.airgap.yml: runs pre-loaded images (image:, never build:),
drops the ./plugins bind mount (which would mask the image's baked-in plugins
with an empty host dir and load zero plugins at an image-only site), and adds
a one-shot migrate service (db upgrade + plugin upgrade-all + seed) that api
waits on via service_completed_successfully, so 'up -d' brings a working site.
- docs/DEPLOY-AIRGAP.md: full runbook (build, transfer+verify, load+run, admin,
verify, upgrade, troubleshooting), incl the Zscaler in-build cert caveat.
- .env.example: IMAGE_TAG for the air-gap compose to pin the loaded image tag.
A geenforce.fetch token can now be pinned to specific manifest scopes so a
fleet-wide key (a display's, delivered by DSC or baked into the image) is not a
skeleton key for the whole content store. NULL binding = unrestricted, so every
existing service token keeps working.
Core:
- ApiToken.resourcescopes column + resourcescopelist property (migration
7d30_apitoken_resourcescopes; NULL = unrestricted).
- apitokens API create/update accept + persist an optional resourcescopes list
(a resource-name allowlist; not permission-catalog names).
- New contract helper authorized_service_token(scope): same check as
service_token_authorized but returns the ApiToken so a plugin can read its
binding. Contract 0.14.0 -> 0.15.0; also export SupportTeam.
GE-Enforce enforcement:
- get_manifest: a bound token requesting a scope outside its allowlist -> 403.
- get_payload: a bound token may only pull a blob its own scope(s) reference
(service.blob_referenced_by_scopes); anything else -> 404 (no hash probing).
- Decorator stashes the authorized token on g for the route to read.
Also fixes a pre-existing contract-surface violation: the printers/printedparts
alert helpers imported shopdb.core.models / shopdb.extensions directly; now
via shopdb.api (SupportTeam newly exported). Docs: GE-ENFORCE-DISPLAY.md
provisioning note, PLUGIN-HOOKS.md, CLAUDE.md.
9 new resource-binding tests; full suite 1131 passing.
Per decision: displays need none of the fleet-wide common scope's software, so
the gea-shopfloor-display scope carries everything it enforces and does not
inherit common. This avoids repackaging common's SMB-backed payloads for a
share-less display.
- Invert the client common-merge switch: -NoCommon (default-on) becomes
-IncludeCommon (default OFF). A scope now enforces alone unless opted in.
The capability stays for a future share-less non-display PC; displays omit it.
- Drop the common SMB-payload audit + inheritance sections from the display
seed comments and docs (GE-ENFORCE-DISPLAY.md); document self-sufficiency.
- GE-ENFORCE-CLIENT.md: common-scope inheritance is now opt-in.
Get GE-Enforce closer to running on credential-less Intune/Entra display PCs
that pull manifest + payloads over HTTPS instead of SMB.
Server (plugins/geenforce/api/routes.py):
- Rate-limit + 512MB served-size ceiling on GET /payload/<sha256> (reuses the
login limiter's cache pattern, config-overridable via GEENFORCE_PAYLOAD_*).
- New tests: payload hardening, manifestblobs model-vs-migration parity, and a
report-contract test locking the lowercase per-entry report keys.
PS client (plugins/geenforce/client/):
- Fix New-ShopdbReport per-entry key casing to lowercase (name/action/selfhealed/
exitcode/message) to match what the server reads; the engine emits PascalCase.
- Enforce TLS 1.2 in the network functions.
- Fetch + merge the fleet-wide common scope alongside the pctype scope
(pctype wins on conflict; -NoCommon opt-out).
- Normalize whatever the engine returns into a well-formed summary.
- Make the empty-cache fail-safe observable: event-log entry + report ping
instead of a silent exit 0.
Manifest (plugins/geenforce/seed_display_scope.py + docs/GE-ENFORCE-DISPLAY.md):
- Seed a gea-shopfloor-display scope: 4 Edge kiosk drift-heal registry entries
+ 1 data-driven dispatcher (Dashboard/Lobby/3DPrintRoom via display-type.txt).
Kiosk EXEs stay image-baked; the manifest heals policy/config drift only.
- Documents the common SMB-payload audit (entries needing http/inline before a
share-less display can inherit common).
Migration registry (shopdb/plugins/alembic_template.py + test):
- Register the pre-existing manifestblobs and the new printersupplyalerts tables
in PLUGIN_TABLE_OWNERS; update EXPECTED_HEAD_REVISION for geenforce (0002blobs),
printers (0002supplyalerts), and printedparts (0004txnrev) which had drifted.
Poll Zabbix for toner levels on a schedule and email/webhook on a downward
crossing. Warning fires at or below the warning threshold (default 5%),
critical at the critical threshold (default 0%); both thresholds are settings.
State lives in printersupplyalerts so an alert fires once per crossing and
re-arms after a refill.
Recipients mirror the printedparts pattern: plugin-scoped shopdb users +
roles + free-text emails (falling back to the site alert_recipients), and a
chosen support team's webhook (falling back to the site alert_webhook_url).
- PrinterSupplyAlert model + migration printers0002supplyalerts
- alerttier(remaining, warning, critical) + check_supplies poller
- flask printers check-toner-alerts CLI (run via scheduled task/cron)
- printers alert settings + Low-Toner Alerts settings page
- 7 tests: tier boundaries, once-per-crossing + re-arm, toner-only scope,
custom thresholds, support-team webhook routing
Support teams gain a webhookurl (migration 7d29 + API + settings-page field), so
a team is a notification target. send_webhook(url=) lets a caller override the
site default with a team's webhook. Printedparts gains a 'alert support team'
setting (printedparts_alert_supportteamid) + selector on its settings page;
low-stock alerts post to that team's webhook, falling back to the site
alert_webhook_url. Email leg unchanged. Same pattern extends to other alerting
plugins (printers low-toner next).
The webhook + site_base_url settings existed in the backend but the Email
Settings page had no inputs (hardcoded fields), and the settings composable's
reactive map didn't include the keys so they never loaded. Add the three inputs
(site base URL, webhook URL, webhook format select) and register the keys.
send_webhook(title,text) posts alerts to an optional webhook (Teams Incoming
Webhook / Workflow, or generic JSON) via alert_webhook_url + alert_webhook_format
settings; send_alert fans out to it alongside email; exposed on shopdb.api
(0.13.0->0.14.0, PLUGIN-HOOKS synced); low-stock posts on its custom-recipient
path too. Also: recent-transactions table shows the consumed print-file revision.
Kiosk now displays the scanned revision (badge, quantity, and done screens) so
the operator sees which revision they checked out - it was already recorded on
the take, just not shown. Low-stock email now uses the gage lab tag (was the
internal item code) and links to the item page when site_base_url is set (new
core setting, category email; emails have no request context to derive the URL).
Label switches from CODE128 to a QR encoding 'TAG|rev' (gage lab tag + latest
print-file revision), so a physical part carries which revision it was printed
from - short payload stays low-version + reliable at 0.5in (margin quiet zone,
EC M, no logo). Item exposes latestrevision; kiosk strips the |rev to resolve
and records the scanned revision on the take (migration 0004 adds
printeditemtransactions.revision) for traceability of which rev was consumed.
Manual entry records a null revision.
The time fields were hidden for the employee types (recognition, recert); show
them for every type. Backend already honors start/end on create and update and
only auto-fills the end when left blank (recognition = next 8 AM, recert = two
weeks), so nothing server-side changes.
Recognition edit hid the time fields (grouped with Recertification), so start/end
could not be adjusted even though the backend honors them. Show the time fields
for every type except Recertification (due-date driven); Recognition end still
auto-fills to the next 8 AM reset when blank.
Also GE-Enforce B2 client (HTTPS payload consume): ShopdbEnforceClient.psm1 gains
Get-ShopdbPayload (fetch by sha256, verify, cache) + Resolve-ShopdbPayloads
(rewrite http/inline entries to local staged files so the engine installs from
local, no SMB); Invoke-ShopdbEnforce resolves payloads before running the engine;
importer parses PayloadSource/PayloadSha256/PayloadRef. VM-verified: a SYSTEM
Windows client fetched a payload over HTTP by hash, hash matched.
Lets share-less (Intune/local-account) PCs pull installers the manifest
references over HTTPS instead of SMB - the general capability the whole fleet
migrates toward. New ManifestBlob registry (migration 0002) with bytes on disk
at instance/geenforce/payloads/<sha256> (deduped by content); service.store_blob
+ blob_path; client-facing GET /api/geenforce/payload/<sha256> (geenforce.fetch
token, ETag=hash, serves the blob store or an inline DB payload by hash). The
serializer now emits PayloadSource/PayloadSha256/PayloadRef for http/inline
entries only (smb entries round-trip unchanged - parity green). CLI
'flask geenforce add-payload <file>' registers a blob and prints its sha256.
This is the shopdb half (B1); the PS client/engine fetch is B2.
One 'display' image resolves what it shows from its own IP, like the existing
visitor-location BU mapping. Extend DashboardDefault with displayrole
(dashboard|lobby|partskiosk; migration 7d28, businessunitid now nullable since
only the dashboard role needs one) + a role->path map. New unauthenticated
GET /api/dashboarddefaults/display-role returns {role, path, businessunitid}
for the caller IP. Settings UI gains a Display selector, showing the business
unit only for the dashboard role.
Switch the dashboard to Verdana (system font on the Windows kiosks - built for
on-screen distance reading, zero bundle) and drop the Archivo package. Restyle
the event card's type-color indicator to span the whole left edge of the card
(old-site style) instead of a small pill.
A wrapped (2-line) name made its tile taller and shifted the whole recert grid.
Reserve a fixed two-line height on .recert-name (line-clamp 2 + ellipsis) so
every tile is the same height regardless of name length.
Bundle @fontsource-variable/archivo (air-gap safe) and apply it to the shopfloor
dashboard only - a sturdy grotesque built for signage/displays, more legible
from across the shop than Inter. Rest of the app stays on Inter.
Shrink the fit surface to 16:9 1600x900 so the fit-scaler upscales the whole
board ~1.2x (more readable from across the shop) and it fills a 1080p TV exactly.
Footer was position:fixed inside a transformed ancestor so it floated at the
content bottom; make it in-flow with margin-top:auto (flex column) so it pins to
the bottom of the surface = the screen bottom.
The board was a fixed-pixel layout with an internal overflow-y scroll, so on a
TV (no scrolling) content past the fold was unreachable. Wrap it in a fixed
1920-wide surface and scale it to fill the viewport (ResizeObserver + resize),
so the whole board is visible edge-to-edge at any resolution. Drop the
.dashboard-content max-height/overflow scroll.
SlideManager rendered <img :src=slide.url> raw, so the root-relative
/api/slides/img/... path 404'd under the /ops subpath mount (the /tv display
already wrapped withBase; the manager did not). Wrap the thumbnail in withBase.
Also switch the multi-column grid to a single-column list with order numbers so
the top-to-bottom play order is clear to arrange.
TVDashboard hardcoded surface=lobby, so only the lobby display was reachable.
Read the surface from route meta (/screensaver -> shopfloor) or a ?surface=
query, defaulting to lobby. Adds a /screensaver toplevel route so the shopfloor
screensaver surface can be displayed on a kiosk.
The badge step card still showed the internal item code; show the gage lab tag
(fallback to item code) to match the label and list. The take still posts the
item code as the stable identifier.
Kiosk item lookup shows a fixed "WJ" prefix addon so operators type only the
number off the label. The label page - and the detail "Part Label" button,
renamed from "Bin Label" - now barcodes and prints the gage lab tag, falling
back to the internal item code when a part has no tag assigned.
Model image URLs are root-relative (/api/models/image/...), so on an /ops
subpath deploy the raw <img src> resolved to the server root and 404'd. Wrap
every model-image src in withBase(): machine/printer/PC/network detail heroes,
the models settings preview, and the machine-badge / asset-label print pages.
withBase leaves external http(s)/data URLs untouched.
The bin step hid entry behind a "Type the number" link, and a plugged-in
keyboard could not drive the visible fields. Replace the hidden wedge input
with one visible, always-focused input per step that a wedge scanner, a
physical keyboard/numpad, and the on-screen keypad all feed; Enter submits.
inputmode="none" keeps the OS soft keyboard from popping on a touchscreen.
The list showed the internal auto-minted itemcode; the gage lab works from the
WJRP gage lab tag. Show gagelabtag as the primary identifier, falling back to
itemcode when a row has no tag assigned.
Role badges rendered gray for everything except admin, with no way to tell
roles apart. Add an optional color per role, matching how statuses and types
carry one: new roles.color column (migration 7d27_roles_color), color threaded
through the role API and the user serializer, and a ColorSwatchPicker in the
role editor. Badges use the role's color with contrast-aware text and fall
back to the old admin/gray classes when unset.
The org IP allow list blocks GitHub-hosted runner IPs (checkout 403), so
point all jobs at the self-hosted arc-runner-set. Drop the rsync dependency
in build-site.sh (cp + bytecode prune; the ARC runner image has no rsync)
and remove the migrations-mysql job - ARC/Kubernetes has no service
containers, so that MySQL 8 coverage stays on the internal CI.
Shopfloor Dashboard is a core view but its content is entirely
notificationsApi.getShopfloor() + the calendar (both owned by the notifications
plugin). Without notifications the display is empty, so gate the Displays link
on the notifications/calendar route being staged. On a site without it the link
- and the Displays header when nothing else is present - drops.
The Displays section is hardcoded in AppLayout (not plugin nav). TV Slideshow
(/tv, slides) had no gate at all and Parts Kiosk (/parts-kiosk, printedparts)
was gated on isPluginEnabled - which a registry copied from a full site reports
true even when the plugin was never staged, so both showed on a lean site and
dead-ended blank. Now each Displays link (Shopfloor, TV Slideshow, Parts Kiosk)
is gated by whether its route is registered in this build (router.getRoutes),
and the Displays header hides when none are present. Route existence is the true
'is it in this build' test.
settingsNav.js hardcodes plugin settings (PC Access Protocols, Machine Types,
VLANs, Employee Directory, ...). A lean per-site build only stages the chosen
plugins' settings routes, so the settings rail showed cards for absent plugins
that dead-ended on a blank page. useSettingsCatalog now filters the catalog to
cards whose target route is registered in this build's router (router.getRoutes),
dropping now-empty groups. Generic - gates every settings card by staged routes
with no per-plugin logic; full builds keep every card. Found testing a live
machines+printers lean site.
A frontend dir under plugins/ with no manifest.json is a CORE feature, not a
per-site plugin - applications is one (backend is shopdb/core/api/applications.py,
nav is advertised as core in dashboard.py). stage-frontend.mjs treated it like
a plugin and dropped it under SITE_PLUGINS, so a lean site showed the core
Applications nav item but had no route for it -> blank page. Now manifest-less
frontends always stage regardless of SITE_PLUGINS; SITE_PLUGINS selection applies
only to real plugins. CI lean-build job asserts ApplicationsList ships in a lean
bundle. Found while testing a live machines+printers lean site.
A lean site still gets every plugin's tables from the shared core Alembic
baseline. prune-schema drops the tables of plugins not installed on this
site, leaving core + chosen-plugin tables, with no edit to any released
migration (the relocate-into-plugin-baselines alternative would mean
rewriting ~15 released core migrations for a cosmetic gain - see ADR-014).
- shopdb/plugins/cli.py: prune-schema command. Dry-run by default; --yes to
execute; refuses non-empty tables without --force. Drops by table name (no
plugin import) so it works on a lean image. MySQL: private AUTOCOMMIT engine
(db.engine's pooled connections sit idle-in-transaction in a CLI context and
would deadlock the DROP on a metadata lock). SQLite: db.engine, restoring the
prior foreign_keys pragma so the StaticPool connection is not left changed.
- tests/test_plugin_prune_schema.py: drop-only-not-installed, full no-op,
refuse-non-empty, force-drops-non-empty.
- docs/DEPLOY.md: lean provisioning step after upgrade-all.
- ADR-014 ACCEPTED; index updated.
Verified on MySQL: full install then prune = no-op (86 tables); lean install
(machines+printers) then prune drops the other 19 plugin tables; second run
no-op. Full suite 1077 passed.
Cross-plugin FK blocker ADR-013 cited is already resolved: the FKs into
machines were held only by dead legacy tables (machinerelationships,
printerdata, installedapps, communications.machineid) that existing
migrations 7a01/7c01 already drop. No live plugin table hard-FKs another
plugin. Schema-lean is unblocked.
Enabling change: create_plugin_tables now skips already-existing tables
(idempotent) so a plugin anchor can create its tables on a fresh lean
install and no-op on a database that has them from the pre-cutover
baseline. The load-bearing baseline lift is staged as ADR-014 Phase 2.
New CI job builds a lean site (machines + printers) via build-site.sh and
asserts omitted-plugin code (PartsKiosk, ManifestEditor, USBLabelBatch,
KnowledgeBaseDetail) is absent from the bundle while chosen-plugin code is
present, and that only chosen plugin dirs stage into the backend. Locks the
lean-build guarantee so a future change cannot silently pull an unchosen plugin
into a per-site build.
The lean-build endgame: a site ships carrying only the plugins it chose.
- scripts/build-site.sh: reads a site profile, resolves the hard-dependency
closure from manifests, builds the frontend with SITE_PLUGINS (stage-frontend
carries only those plugins), and stages a backend tree of core + only the
chosen plugin dirs. An unchosen plugin is in neither the bundle nor the tree.
- Core lazy-import guard: `flask seed demo` hard-imported the 5 asset subtype
models, which would crash a lean build missing any of those plugins. Now
guarded (a missing model skips its demo section).
- test_lean_build_guards.py: statically asserts NO core (shopdb/core, shopdb/cli)
import of a plugin is unguarded - a lean build omitting that plugin would
otherwise crash. 0 unguarded today.
Pilot verified: a lean build (machines + printers) carries only machines +
printers code - PartsKiosk / ManifestEditor / USBLabelBatch / KnowledgeBaseDetail
/ EmployeeDirectory are absent from the bundle, and only machines/printers plugin
dirs stage into the backend. (Sidebar labels for absent plugins remain - the
accepted small plugin-aware core remainder.) Guard test + naming green.
The naming/style check now fails a plugin frontend (plugins/<name>/frontend/)
that imports with an escaping ../../ or another plugin's path. Plugin frontends
must reach core only through the @/ alias and otherwise import only their own
tree, so a per-site build can drop a plugin cleanly. All 14 plugin frontends
pass.
core.js still routed plugin-owned pages directly. Extracted all 11 into the
owning plugin's route file + moved their views into plugins/<name>/frontend/:
- computers: reports/pc-relationships, settings/pctypemapping
- printers: reports/toner, settings/printertypes, settings/zabbix (toner/supply
monitoring)
- machines: settings/machinetypes
- network: settings/networktypes
- warranty: settings/dellwarranty
- slides: settings/slides (its route file gains a default export; it was
toplevel-only)
- employees: NEW plugin frontend (employees/:sso + settings/employeedirectory) -
employees had no route file before; its pages lived only in core.js.
core.js now holds only core routes; all 14 bundled plugins are self-contained
under plugins/<name>/frontend/. Verified live: the extracted Machine Types
settings page renders in the settings rail from the machines plugin frontend.
Build + 58 vitest + naming green.
Relocate warranty, measuringtools, network, printers, usb, notifications,
computers, and slides into plugins/<name>/frontend/. Each plugin's views are
pulled from wherever they lived (own dir, plus the shared views/settings/,
views/reports/, views/print/ dirs, and top-level views) into the plugin's
frontend/views/, and its route file becomes the self-contained routes.js.
Handled the messy cases:
- computers: name mismatch (its views live in views/pcs/) - moved by following
the route file's own imports, so the dir name did not matter. Its OS/access-
protocol/PC-type settings views move with it (only computers.js routed them).
- network: NetworkHub's sibling sub-views (NetworkDevicesList, SubnetsBrowse,
not directly routed) moved too so its `./` imports resolve.
- printers: the qrLogo helper is SHARED with core AssetLabel, so it stays in
views/print/ and PrinterQR imports it via @/views/print/qrLogo.
- slides: route file is toplevel-only (TVDashboard); SlideManager stays core
(core.js routes /settings/slides).
frontend/src/views/ now holds only core views; frontend/src/router/routes/ holds
only core.js. All 13 plugins are self-contained under plugins/<name>/frontend/.
Verified live: Network (hub + moved sub-views), Computers (name mismatch),
GE-Enforce (helper), printedparts all render from their staged frontends. Build +
58 vitest + naming green.
Relocate applications, geenforce, knowledgebase, and machines - each owns only
its own views dir, so a clean move to plugins/<name>/frontend/ (views/ +
routes.js, core imports rewritten to @/). geenforce's entryForm.js helper + its
vitest spec move with it (ManifestEditor imports it as a sibling).
Machinery fixes this batch surfaced:
- routes.gen.js codegen uses namespace imports (import * as p_x). A route file
without a `toplevel` export is undefined on the namespace instead of a strict-
ESM missing-binding build error.
- vitest gains a `pretest` stage so plugin-frontend specs (now under
plugins/<name>/frontend/) run from their staged copy in src/.plugins-staged/.
Verified live: GE-Enforce (the most complex, uses the entryForm sibling helper)
renders fully from its staged frontend. Build + 58 vitest + naming green.
The staging step that makes lean per-site frontend builds possible, plus the
first plugin relocated as the pilot.
- scripts/stage-frontend.mjs: copies each chosen plugin's plugins/<name>/frontend/
into frontend/src/.plugins-staged/<name>/ and codegens routes.gen.js. Plugin
selection via SITE_PLUGINS (comma-separated); empty = all plugins that have a
frontend/ (the full build). Wired as npm predev/prebuild; outputs gitignored.
- Router imports routes.gen.js and merges staged routes with the in-tree
./routes/*.js glob - dual-location during the transition.
- printedparts relocated: its 6 views (list/detail/form/kiosk + the settings and
labels views from the shared dirs) moved into plugins/printedparts/frontend/
views/, core imports rewritten to the @/ alias; routes.js is the self-contained
route module. Its old in-tree route file is removed.
Also fixes a crash the previous commit (37c764b) shipped: slides.js exports only
`toplevel` (its child routes live in core.js), so the router's
flatMap(m => m.default) produced an undefined child and threw
"Cannot read properties of undefined (reading 'path')" at load - the whole SPA
went blank. Guarded with `m.default || []`. (The earlier "print pages are blank"
reading was this crash, not page nature.)
Verified live: /machines renders again; the relocated /printedparts list renders
identically from the staged plugin frontend; SITE_PLUGINS=machines excludes
printedparts from routes.gen. Build (via npm, runs stage) + vitest + naming green.
Core-router surgery (the Phase 4 prerequisite for lean builds): index.js
hardcoded six plugin-owned full-screen routes (parts-kiosk, TV, printer-qr x2,
usb-labels, printedparts-labels), so pruning any of those plugins broke the SPA
build on an unresolvable import. The router now also collects a `toplevel`
named export from each plugin route file (alongside the existing default =
AppLayout children) and spreads it into the top-level routes. Each of the six
routes moved into its owning plugin's route file (printedparts, printers, usb,
slides); index.js keeps only the core print pages that span asset types
(machine-badge, asset-label, asset-label-batch).
index.js now references zero plugin view components. Verified: all six route
paths are present in the built bundle and the moved routes resolve exactly like
the unchanged core print routes. Build + vitest + naming green.
Wires the ADR-010 get_map_overlays hook into the floor map so a plugin decorates
markers as JSON, no map code. ShopFloorMap fetches /api/pluginui/map-overlays,
then each overlay's endpoint (per-asset [{assetid, color, label}]), joins by
assetid, and draws a ring or badge circleMarker on matching markers plus a
legend entry - all as extra Leaflet layers cleared and redrawn with the markers.
Aligned the measuringtools calibration overlay endpoint to the documented
contract: it now returns {assetid, color, label} (was {calibrationstatus,
statuscolor}) and only decorates due/overdue tools.
Additive + guarded (assetid null check, per-endpoint try/catch, cleanup on
re-render), so the map degrades to no decorations on any failure. Verified: the
overlay endpoint serves the contract shape, the map renders without error, and
the frontend builds. A populated badge needs a site that actually places
measuring tools on its map (this dataset places none). 38 measuringtools/pluginui
tests, 58 vitest, build + naming green.
Global-search rows built the plugin detail URL from a hardcoded url_map of
plugin routes in core. Now core prefers a plugin's declared
get_asset_presentation route (ADR-010), substituting the core assetid via the
plugin's by-asset resolver; types that have not declared fall back to the legacy
id-keyed map, so nothing breaks. Measuring tools (which declare the route) link
through it now; machines/PCs/printers/network migrate off the hardcode as they
add a by-asset route + declaration. Presentation map is collected once per
search (cached on flask.g). 2 consumer tests; 26 search tests green.
Rolls the generic renderer into the remaining four detail pages (PCDetail,
PrinterDetail, NetworkDeviceDetail, MeasuringToolDetail), replacing the
hand-composed <WarrantyPanel> with <PluginAssetPanels>. The warranty hero badge
(useWarrantyBadge) stays on the pages that show it; MeasuringToolDetail dropped
its now-unused warranty composable usage.
WarrantyPanel.vue is deleted - warranty now renders entirely from its
get_asset_panels JSON declaration through the generic renderer. Verified live on
a PC with a warranty: the card is identical to the old bespoke panel (vendor
title, Expiring Soon status badge with color, servicelevel/ends/tag meta, manage
link) with no warranty-specific frontend code. Build clean, 58 vitest, naming green.
Wires the ADR-010 get_asset_panels hook to a generic frontend renderer so a
plugin adds detail-page UI as JSON, no Vue. This is the Path A foundation that
lets simple plugins ship UI without a frontend build.
- components/PluginAssetPanels.vue + pluginAssetPanels.js: fetches
/api/pluginui/asset-panels for an asset, then each panel's data endpoint, and
renders by mode: list (title + status badge + meta lines via a field map),
keyvalue, table (declared or inferred columns), badge. Pure mapping logic is
in the .js module and unit tested (9 specs), same pattern as entryForm.js.
- New 'list' render mode with a declarative field map (title/badge/meta),
documented on the hook in base.py.
- Warranty migrated to it: get_asset_panels now declares a 'list' panel + map
that reproduces WarrantyPanel's output (vendor title, status badge with color
+ label map, servicelevel/ends/tag meta, manage link) with zero
warranty-specific frontend code.
- MachineDetail swapped from <WarrantyPanel> to <PluginAssetPanels> (pilot); the
hero warranty badge is unchanged. Verified end to end: the API serves the list
panel + map and the warranty rows; the page renders without error.
Rollout of the other 4 detail pages (PCDetail, PrinterDetail, NetworkDeviceDetail,
MeasuringToolDetail) and the map-overlays / asset-presentation renderers are
follow-up Phase 3 commits. 58 vitest, build clean, 1067 backend pass, naming green.
Fourth review found the last import-path bypass: the is_dir() branch returned
None for a name whose dir has no __init__.py, without checking a same-name
sibling file. FileFinder loads a file over an init-less namespace dir, so an
attacker could overwrite a signed foo.py with malicious bytes, mkdir an empty
foo/ next to it (PROVENANCE untouched, still verifies), and any import of that
name ran the unverified foo.py - RCE with only plugins/ write access.
Fix: the dir-with-no-__init__.py branch no longer returns early; it falls
through to the leaf .py hash gate and the non-source refuse check. Invariant:
find_spec returns None for a plugins.* name ONLY where FileFinder would also
find nothing on the same __path__.
Everything else was confirmed sound this round: the owned plugins root, exec of
exact verified bytes (never .pyc/.so), the extension/bytecode refusal, plugin.py
read-once, the provenance signature gate, dev-exemption scoping, and #3/#4.
Symlink, suffix-ordering, cache-lifecycle, and loader-internal angles cleared.
2 regression tests (tampered .py + sibling dir; unsigned .py + sibling dir). All
13 bundled plugins still load under enforcement; 1067 pass, naming green.
Third review found the meta_path guard leaked exactly where it delegated to the
stdlib import system:
1. Non-.py submodules (CRITICAL). When a name had no dir and no .py, find_spec
returned None and the stdlib loaded a planted .so (ExtensionFileLoader) or a
sourceless .pyc unverified - an attacker deletes a signed .py and drops a
same-named .so with arbitrary init code, run on a normal request via core's
`from plugins.<name>.models import ...`. The guard now refuses any name for
which a non-source importable candidate (EXTENSION_SUFFIXES + BYTECODE_
SUFFIXES) exists on disk; None is reserved for genuinely-absent modules.
2. Top-level plugins/__init__.py (CRITICAL). It is in no plugin's provenance,
is attacker-writable, and Python runs it before any guarded submodule. The
guard now owns `plugins`: it execs an EMPTY package body (search points at
the plugins dir), so an overwritten plugins/__init__.py never runs.
Also: specs are built with spec_from_file_location so loaded modules get
__file__/__path__ (Flask blueprint root paths need it) while the loader still
execs the verified in-memory bytes - never re-reading the file.
Verified end to end: under PLUGIN_REQUIRE_SIGNED with all 13 bundled plugins
stamped, the app boots and loads every plugin through the guard; a tampered
plugin file is refused at load. 4 new guard tests (planted .so, sourceless
.pyc, absent-module defer, neutralized package root). Prior fixes #3/#4
confirmed still sound by the review. 1065 pass, naming green.
A re-review showed the previous "single import choke point" claim was wrong:
`plugins` is a normal importable package, so core request handlers that do
`from plugins.<name>.models import ...` never passed through the loader and ran
unverified - an attacker who dropped a file into plugins/<name>/ got arbitrary
in-process code execution on an ordinary HTTP request (and a planted .pyc ran
from cache). Gating load_plugin_class covered only plugin.py, one path of many.
Fix: importguard.py installs a sys.meta_path finder (under enforcement) that
intercepts EVERY plugins.<name>.* import, verifies the plugin's signed
provenance once, then verifies each module file against it and execs the exact
bytes it hashed - read once, compiled, exec'd, never a .pyc, never a re-opened
file. This closes the submodule bypass and the planted-bytecode read, and the
read-once exec closes the verify-vs-exec TOCTOU on the import path. The import
system, not one method, is the real choke point.
- init_app installs the guard when PLUGIN_REQUIRE_SIGNED, clears it otherwise.
- load_plugin_class now verifies plugin.py from a single read and execs that
buffer (finding #3 on that file); its submodule imports flow through the guard.
- docs: stamp-bundled must cover every plugin dir present (a disabled plugin's
module can be imported by core); recommend a read-only plugins/ owned by the
deploy user as defense in depth (closes the residual migrate-time race an
attacker with concurrent write could otherwise attempt).
Earlier review's fixes#3 (migrate code paths) and #4 (shelf content binding)
were confirmed sound and are unchanged. 7 import-guard tests (submodule verify,
tamper, unsigned refused, planted .pyc ignored, real import through the guard,
install/uninstall). 1061 pass, naming green.
An adversarial security review of the Phase 2 trust model found four real
bypasses (two remote-triggerable to in-process code execution). Root cause for
three: the set of bytes verification covered was smaller than the set that
determined execution. Fixes:
1. Bytecode-cache blind spot (CRITICAL). verify_dir excluded __pycache__/.pyc,
so a planted cache ran while escaping the hash map. verify_dir now flags any
bytecode as an unexpected file; the loader strips bytecode before verify and
imports under sys.dont_write_bytecode, so only verified source executes.
2. Unauthenticated verify-at-load bypass (CRITICAL). load_plugin_class imported
plugin.py with no gate, reachable via discover_available / an anonymous GET
/api/plugins. The verify+strip gate moved INTO load_plugin_class - the single
import choke point every path flows through - so an unsigned/tampered plugin
is never imported. discover_available skips a refused plugin instead of 500.
3. Ungated migration entrypoints (HIGH). downgrade_plugin and get_current_head
(ScriptDirectory imports version modules) ran plugin code with no check. All
alembic-invoking methods now pass through _verify_ok (strip + verify) first
and run under no-bytecode.
4. Revocation/content bypass (HIGH). The signed index bound a filename, not
content; adopt did not bind the delivered bytes to the resolved version, so
revoked bytes could be served under a live filename. The index now records a
per-artifact SHA-256; adopt verifies the on-disk digest and requires the
artifact's own signed manifest version to equal the resolved version.
Enforcement stays default-off; strip/no-bytecode run only under enforcement, so
the unsigned path is unchanged. 6 regression tests (planted bytecode, the
discover import path, downgrade gate, version-swap). 1054 pass, naming green.
Completes the marketplace security model. Verification stops being advisory:
a plugin only loads or migrates when its tree matches a trusted signature, and
plugins are pulled from a signed shelf with anti-rollback and revocation.
Enforcement (default OFF - existing deploys unchanged):
- verification.py PluginVerifier, shared by the loader (verify-at-load, before
plugin.py is imported) and the migration manager (verify-at-migrate, before
any DDL). Fail-closed: an unsigned/tampered/wrong-key plugin does not run.
- Gated by PLUGIN_REQUIRE_SIGNED. PLUGIN_DEV_TRUST_DIRS exempts named dirs but
only under DEBUG/TESTING; production ignores it.
- flask plugin stamp-bundled writes provenance into in-tree plugins so
verify-at-load applies to bundled plugins too (image build step).
- tier:core manifest guard: uninstall/disable refuse a core-tier plugin.
Shelf (shelf.py):
- Signed shelf-index.json (+ .sig): monotonic serial (a site refuses an older
index - anti-rollback), revoked list carried across builds, per-entry
version/tier/core_version for browse. Index is a browse layer only; adopt
reads security-bearing fields from the verified artifact.
- flask plugin shelf-build / shelf-list / adopt / audit. adopt verifies index +
artifact (signature + every file hash), unpacks to staging, re-verifies, then
atomically moves into place and installs+enables the closure. Refuses a
downgrade without --force-downgrade. Anti-rollback serial stored in
instance/shelf-state.json.
- config PLUGIN_SHELF_DIR; the app only reads the folder, never speaks a
network. .env.example + docs/PLUGIN-SIGNING.md document the flow.
22 tests: verifier policy (off / no-keys / signed / tampered / wrong-key /
dev-exempt), verify-at-load + verify-at-migrate integration, tier guard, index
sign/verify + tamper/wrong-key, serial state, revocation, version resolution,
verified atomic unpack + tamper refusal. Live-smoked keygen->pack->shelf-build
->list->adopt->audit + serial guard. 1050 pass, naming green.
Packaging + provenance for the plugin marketplace. No runtime behavior change
yet - verification is available on demand; enforcing it at plugin load/migrate
and pulling from a shelf are Phase 2.
- signing.py: ed25519 key pairs + provenance. Provenance is a sorted per-file
SHA-256 map plus metadata; the detached signature covers the exact
serialized provenance bytes, so verifying is re-hash files, re-serialize,
check signature. verify() accepts any of several trusted keys (rotation).
Uses cryptography (already a dependency).
- packaging.py: pack() builds a signed <name>-<version>.shopdbplugin (zip +
PROVENANCE.json + PROVENANCE.sig). verify_artifact()/verify_dir() re-hash
and check the signature, and flag a tampered file, an unexpected file, a
wrong/absent key - all fail closed.
- CLI: `flask plugin keygen` (publisher key pair), `flask plugin pack <name>
--key` (validates then signs), and `flask plugin validate` extended to a
signed artifact by path (--pubkey, else PLUGIN_TRUSTED_KEYS).
- config PLUGIN_TRUSTED_KEYS: os.pathsep-separated public-key PEM paths,
delivered with the site config, never read from the shelf. .env.example
documents it.
- docs/PLUGIN-SIGNING.md: curator flow (keygen offline, review, pack, publish,
pin keys, rotate).
The signature proves an artifact is exactly what a curator signed, not that the
code is safe - human review before signing is the control. 11 tests: sign/verify
round trip, wrong key, provenance excludes noise, serialize determinism, pack +
verify, tamper -> hash mismatch, extra file, no-key fail-closed, verify_dir.
1028 pass, naming green.
Additive, zero-risk-to-running-sites prep for the plugin catalog. No
distribution or lean-build behavior yet; fixes latent bugs and adds the
declarative + validate tooling later phases build on.
Fixes:
- upgrade_all_plugins iterates registry.get_all(); only adopted plugins are
migrated. Removes the phantom hasattr(registry, 'list_installed') probe
that always fell through to migrating every folder on disk (unadopted DDL
ran with full DB rights on every deploy).
- Reverse-dependency checks on uninstall/disable read dependencies from the
manifest on disk via _installed_dependents, so an installed-but-unloaded or
disabled dependent is counted. Uninstall blocks on any installed dependent;
disable blocks on an enabled dependent.
- _sort_by_dependencies detects a dependency cycle (back edge in the DFS) and
raises PluginDependencyError instead of looping or dropping a plugin.
New:
- flask plugin validate <name>: manifest loads + name match, manifest-schema
check, core_version admits the framework contract, declared dependencies
exist on disk. No new dependency (lightweight checker); schema ships in the
package at shopdb/plugins/manifest_schema.json (docs/ is stripped on
publish). The check caught that provides is an object, not an array.
- flask plugin apply-profile <file>: declarative install AND enable of a
chosen plugin set plus its hard-dependency closure, in dependency order,
idempotent. Replaces the hand-ordered runbook sequences that could enable a
plugin that was never installed. deploy/site-profile.example.json template.
- Dockerfile header corrected (all 13 catalog plugins, not "eleven core").
10 new lifecycle tests (reverse-deps from disk, cycle detection, upgrade-all
scope, profile closure, schema, all 13 manifests match schema). 1018 pass,
naming green.
Design record for distributing optional plugins across GE sites: a small
mandatory core plus a catalog of optional plugins, packaged as signed
versioned artifacts, served from a transport-agnostic read-only shelf (a
SharePoint-synced or sneakernet folder - untrusted either way because every
decision-bearing byte is signed), verified at adopt AND at every load and
migrate. Lean per-site builds stage only chosen plugins into the backend
image and SPA bundle.
Status PROPOSED. Grounds the design in the real loader/contract/migration/
frontend code and records defects to fix along the way (upgrade-all
migrating unadopted folders, enable-without-install, reverse-dep checks
blind to unloaded plugins, missing cycle detection and dependency closure,
hardcoded plugin imports in the SPA router). Honest on scope: the frontend
re-org is the long pole (one core-router change plus per-plugin relocation),
not a mechanical move. Phased 0-5 with schema-lean and runtime-JS delivery
explicitly deferred.
Repo-level instructions so GitHub Copilot follows the LOCKED naming rules
(lowercase concatenated DB columns, allowed-acronym list, banned shorthand),
the ASCII-only style policy, and the plugin/migration/contract architecture.
Without this Copilot suggests snake_case columns, em-dashes, and
db.create_all(), which the naming hook and CI then reject. Distilled from
CONTRIBUTING.md; that file stays the authority.
New dev/eval seeder populates a small, broad dataset so a fresh site has
something on every screen: ~25 assets across machines, computers,
printers, network devices, and measuring tools, plus supporting
vendors/business-units/locations, six 3D-printed parts (two below their
low-stock threshold to exercise the alert), and a few relationships for
the map and relationship cards. Idempotent, keyed on a DEMO- assetnumber
prefix; skips the plugin sections that are not installed.
`flask seed demo-clear` removes exactly what it created: bulk-deletes the
DEMO- assets so the DB-level ON DELETE CASCADE drops each plugin subtype
row (per-object ORM delete would try to NULL the NOT NULL child assetid),
after clearing the demo relationships first. Leaves reference data,
settings, users, and any imported rows untouched.
Documented as an optional step in the dev setup guide.
The dualpath_single_machine setting description is 257 chars but
settings.description was varchar(255). On strict MySQL 8 an over-length
insert is a hard error 1406 (Data too long), so `flask seed settings`
failed on a fresh install; older/relaxed MySQL truncated silently and
hid it. Widen the column to TEXT (matches value, already TEXT) via core
migration 7d26.
CI only ran `flask db upgrade` + plugin install, never the seeders, so it
missed this. Add a seed step to the migrations-mysql job so a seeded row
that violates a column constraint fails CI on strict MySQL 8 instead of
shipping.
pymysql needs the cryptography package to speak MySQL 8's default
caching_sha2_password, so 'flask db upgrade' against a stock MySQL 8
failed with 'cryptography package is required'. Make it a real
dependency (dev, prod, CI all connect cleanly) and drop the CI
native-auth workaround that stood in for it.
Prod runs Python 3.13.7, not the originally planned 3.12. Align the
stack: CI both jobs 3.12->3.13, the IIS install runbook and the dev
setup guide to 3.13 (winget Python.Python.3.13). NOTE for whoever
maintains the offline kit: its wheels are still cp312 and must be
regenerated as cp313 before the next air-gapped deploy.
Add a winget block to the prerequisites (Git/Python/Node/VS Code/MySQL
or Docker) so a Windows dev provisions the whole toolchain from one
terminal, with a note that the LTS Node may be newer than CI's 20 and
it does not matter for this SPA (nvm-windows to pin if wanted).
From the Fable/Opus documentation audit (8 confirmed + verified
lab-drift the run's session limit had cut short):
- HIGH: the lab's kiosk _kiosk_find_item block showed the pre-stage-17
row-id resolver as current; replace with the shipped gagelabtag /
numeric-tail resolver, fix the stale 'resolved by row id' prose and
the 'stage-7 code is corrected' note.
- MED: the badge _external_lookup block used dict-only row access that
breaks on a tuple cursor; use the tuple-or-dict form shipped. Split
'&&' command chains (fail in PowerShell 5.1) in the lab.
- LOW/link: the Windows note's [DEVELOPMENT-SETUP] link dropped the .md
and 404'd in four docs; fix. Correct the stage-6a->16a comment and
the lab-stage tag range (..16 -> ..17).
- Leaks: drop /home/camp path from ADR-006, the internal gitea host
from PLUGINS.md.
- Windows: add an mklink junction note for the external-plugin symlink
dev loop.
- CI: prime root to mysql_native_password so pymysql connects to the
MySQL 8 service without the cryptography package (and its kit wheel).
The CI workflow comment named the internal server, and
PLUGIN-EXTERNAL-REPO carried internal gitea clone URLs (it becomes a
public wiki page). Point both at the GitHub home / a generic CI
mention so the publication scrub gate passes and the wiki does not
expose internal infrastructure.
GitHub had no CI, so naming/tests/build were unenforced on the public
mirror. Add .github/workflows/ci.yml mirroring the internal pipeline:
backend pytest, the naming gate, frontend vitest+build, and the
migrations-mysql job that proves a fresh flask db upgrade + every
plugin chain on utf8mb4 MySQL 8 is idempotent. Flip the dev-setup CI
note to reflect it. Add an identical Windows/VS Code convention note to
the four developer docs (venv\Scripts vs venv/bin, $env: vs export,
pointer to DEVELOPMENT-SETUP).
The naming check was documented as an auto-running pre-commit hook,
but .git/hooks is never cloned and no installer existed - a fresh
clone had nothing, and the real enforcement is CI. Say that plainly.
Ship .githooks/pre-commit (LF-pinned) so a dev who wants the local
check can opt in with 'git config core.hooksPath .githooks'; CI stays
the backstop that fails the build on a bad name.
PowerShell commands lead, bash equivalents in comments: venv
Activate.ps1 + execution-policy note, copy/$env:, a PowerShell
plugin-enable loop, and how the bash naming hook runs under Git Bash
(plus the pre-commit hook catching it automatically). The VS Code
Check task gets a Windows variant (venv\Scripts, bash for the .sh).
Pin shell scripts to LF in .gitattributes so a Windows checkout does
not CRLF-corrupt them into 'bad interpreter' failures.
New docs/DEVELOPMENT-SETUP.md: clone-to-first-change onboarding
(Docker fast path, manual venv+Node daily driver, VS Code, the dev
loop, first-change pointer at the plugin lab, troubleshooting). Ship
.vscode/ launch/tasks/extensions so F5 debugs the backend on 5001 and
a task runs both servers; personal settings.json stays ignored. Fix
the README manual path - it ran the backend on the default 5000, but
the frontend dev server proxies to 5001, so nothing loaded; also add
the plugin upgrade-all step and a VS Code pointer.
The gage lab assigns real WJRP asset numbers, so identity splits: the
internal itemcode stays auto-minted and a new optional unique
gagelabtag (migration 0003) carries the lab's number - settable on
create/edit, searchable, and resolved by the kiosk for scans and bare
keypad digits against the numeric tail of either identifier
(unique-match only). The print-files table becomes stacked revision
cards - filename with rev/current badges, one meta line, delete pinned
right - ending the horizontal scroll in that column.
The dark .form-control override used the background shorthand, which
resets a select's background-repeat and position; the dark select rule
then re-added the arrow image without them, tiling it from the top
left. Use background-color in the overrides and restate
no-repeat/position on the select rule.
The milestone workbook becomes a from-scratch guide with the actual
code inline for every core stage: models, the real migration baseline,
read routes and the list page, mutations and minting, the badge
resolver (final mode-aware form), the single-commit ledger invariant,
RBAC gating, both kiosk endpoints with the wedge-input and focus-guard
mechanics, the 1x0.5in label CSS, and the reconcile query. Field
extensions stay summarized against their tags. New section: how to
contribute a plugin through GitHub (branch, stage commits, the three
CI gates, PR expectations, review checklist, and how publication
folds PRs into release commits).
Photos already resolved through the directory at read time, but names
only came from the stored employeename column - empty after a
shopdb-only import, so recertification/recognition cards showed bare
SSOs. New resolve_employee_display_name in the employees plugin
(mode-aware: self-hosted table or external HR) backs a fallback in
both the single-card and split-per-employee paths; stored names still
win when present.
The uploaded-blueprint thumbnails on the map settings page and the
setup wizard used the raw setting value (/api/settings/map-blueprint/
...), which resolves at the server root and 404s under a subpath
mount - while the map itself resolves through blueprintUrlFor and
worked. Wrap the previews in withBase.
Users without a profile photo (and broken photo URLs) show the GE
monogram instead of nothing/initials - sidebar identity, employee
detail hero, and the directory list thumbs; the shopfloor cards
already did this. Document titles become
'<Facility> ShopDB - <Page>' via a router afterEach (facility from
public settings, page label from meta.title or a prettified route
name with spellings for PCs/USB/GE-Enforce/3D Printed Parts/...), so
copied links and browser tabs identify the page.
External HR Picture values are relative paths; the resolver hardcoded
/static/employees/ (which the SPA then mounts under the subpath, e.g.
/ops/static/...), but sites like WJ serve those photos from the
classic EmployeeDBAPP on another URL entirely. New setting
employee_photo_base_url (blank keeps the old behavior; a full URL like
https://host/EmployeeDBAPP/images/ passes through withBase untouched),
declared in the plugin config schema.
Deleting any user who owned an API token or appeared in the audit log
hit the users FK and 500ed - the import's 'importer' account being the
guaranteed case (its PAT plus every audit row the import wrote).
Tokens are revoked outright; audit history is kept but detached
(userid NULL), so the trail survives the account.
The resolver only read the self-hosted directory table, which is empty
at sites running the external HR directory - every kiosk badge fell to
the deny policy. It now branches on employee_directory_mode like the
usb plugin: selfhosted looks up DirectoryEmployee by SSO; external
queries the HR directory via employee_connection, resolving PayNo
badges by their real PayNo column and recovering the employee's SSO.
The plugin registry (instance/plugins.json), uploaded logos, floor
plans, item photos, and print files all write under instance/; with
the app pool at read-only, toggling a plugin in Settings surfaces as
an internal error and every upload fails. Grant Modify in step 7.3
and add the troubleshooting row.
The plugin-lab exemplar built end to end: catalog with photos and
print-file revisions, badge-attributed stock ledger, touch kiosk with
an open decrement-only take endpoint (decision record), 1x0.5in bin
labels, low-stock alerts to users/roles/emails, reports with a
reconcile check, per-plugin migrations 0001+0002, contract 0.13.0
(mailer + User/Role on the plugin surface).
Browsing the catalog (item list, detail, file listings) now requires
authentication plus the view permission, and the /printedparts pages
and the label print page require login. Still deliberately open: the
kiosk endpoints per the decision record, the image serve and file
download (img tags and anchor downloads cannot carry a JWT), and the
reports (product-wide jwt-optional convention). Grant
printedparts.view to the roles that should see the catalog.
The keypad becomes a proper terminal pad: fixed 3-column grid of
rounded square buttons with tabular numerals, press feedback, and
muted Clear/backspace actions. Each manual step (item number, SSO,
quantity) shares one card panel - boxed entry display with placeholder
styling, keypad, and a full-width action button.
The tap-anywhere wedge refocus stole focus from the manual-entry field
the moment it was tapped - the handler now only reclaims focus from
dead space, never from a real control. Manual entry works without a
physical keyboard: badge entry uses the TouchKeypad (an SSO is
digits), and item lookup accepts bare digits resolved by row id - the
digits in a minted code are the id, which also keeps labels printed
under an older prefix scannable after the prefix changes.
printeditemfiles lands as the plugin's first incremental migration
(0002 on the plugin chain - the ADR-008 payoff). Revisions are
append-only per item: upload assigns the next number, records the
uploader from the JWT, enforces an extension allowlist and a 100 MB
cap; download serves the original filename; a permission-gated delete
covers wrong-file mistakes. The detail page gains the revision table
with a current badge. Unique storedfilename is sized 191 so the index
fits MySQL's 767-byte prefix - the per-plugin chain does not apply the
core env's ROW_FORMAT hook.
Alert recipients gain roles: Role joins the 0.13.0 surface, a role
picker on the settings page, and every active member of the selected
roles is folded into the deduped recipient list.
Beside Shopfloor Dashboard and TV Slideshow, opening in a new tab and
shown only while the plugin is enabled - kiosk-style pages get
launched from the Displays group, not the Information nav.
Retire button with confirmation on the detail page (item leaves the
storefront and the kiosk rejects its code; ledger history and label
survive), Restore on retired items, and an Include-retired list toggle
with a badge. Restore is its own permission-gated POST - the generic
update still cannot flip isactive. New codes mint as WJRP0042 style
without the dash; existing codes are immutable bin labels and keep
their form.
Contract 0.13.0 puts the User model on the plugin surface. The
settings page gains a checkbox picker over the user list; selected
users receive low-stock alerts at their account email, merged and
deduped with the free-text address list, inactive accounts skipped,
site alert_recipients still the fallback when both are empty.
PrintedPartsSettings edits the four plugin settings (code prefix,
default threshold, kiosk badge policy, alert recipients) through the
core settings API; the route rides the plugin's router file and the
settings shell nests it into the rail; get_settings_cards contributes
the catalog card while the plugin is enabled.
Contract 0.12.0: send_email/send_alert join the plugin surface (the
mailer was core-only), PLUGIN-HOOKS and status docs updated, manifest
pins the new floor. The alert fires inside _ledger_write only when a
decrement CROSSES the item's threshold - one alert per depletion,
rearmed by restocking above - and is best-effort after the commit so
mail trouble can never fail a take. Recipients come from
printedparts_alert_email, falling back to the site alert_recipients.
on_enable re-seeds settings idempotently so existing installs pick up
new keys. Crossing/rearm semantics proven by test.
The lab is now a build-along mirroring what actually happened: ten
stages, each with the goal, the divergences, a see-it-work check, and
the errors genuinely hit while building (empty Migration error from a
broken model import, the migration-guard KeyError, the missing Lucide
icon, nested-app-context test writes, Decimal sums, and the authz
sweep catching the deliberately open kiosk take). That last one gets
its explicit EXEMPT_ENDPOINTS entry with a pointer to the decision
record - the net stays, the exception is reviewable. Full suite: 993
backend tests, 49 vitest, frontend build, naming hook, all green.
Three jwt-optional endpoints with ?format=csv, merged into the reports
hub via get_reports while the plugin is enabled. The stock report's
ledgerdelta column is the reconcile check: 0 for every item whose
stock moved through the ledger, nonzero for anything that bypassed it
(the hand-seeded dev rows demonstrate the catch). MySQL SUM returns
Decimal - cast to int or the delta serializes as a string.
New public print view at /print/printedparts-labels following the
plugin-owned USB label precedent: multi-select with per-item copies,
CODE128 of the item code via JsBarcode (a QR at this size is at the
edge of scanner tolerance), one label per page on 1in x 0.5in roll
stock via a new @page size. The Detail page's Bin Label button
preselects its item through ?item=<id>; the list header gains a batch
Print Labels button.
Two open endpoints: an item lookup by scanned code and the take POST -
the product's first unauthenticated write, held to the decision
record's bar (decrement-only, badge-attributed server-side, bounded,
physically rate-limited; justification in the plugin README). The
/parts-kiosk route is a full-screen no-auth view beside /shopfloor: a
hidden always-focused input consumes keyboard-wedge scans for
whichever step is active, TouchKeypad (net-new 3x4 grid) takes the
quantity, and a success screen resets after a few seconds. Manual
type-in fallbacks cover damaged labels. Kiosk test proves open access,
the over-take guard, the badge policy, and cache==ledger afterward.
get_permissions declares view/create/edit/delete/restock (seeded on
install/enable and by flask seed permissions); every write route adds
require_permission on top of jwt_required. New test proves
authentication alone is not authorization: a role-less member gets
403 where an admin succeeds.
Badge resolver copied from the USB contract (SSO digits, 0<digits>BZ
PayNo wrap) with names from the employees directory and the
unknown-badge policy setting; deliberately copied rather than
cross-imported so the contract test stays green. Restock and adjust
write the ledger row and move the cached quantity in one commit -
the single-commit invariant every write path must use. Adjust
requires a reason and refuses to drive stock below zero. Detail page
gains Restock/Adjust modals. Seven tests cover minting, the
cache==ledger invariant, badge shapes, policy toggle, and auth.
POST/PUT/DELETE for items: create mints the itemcode from the
configured prefix plus the flushed row id, update refuses
quantityonhand (ledger-managed - restock/adjust arrive next stage),
delete soft-retires. The image upload/serve/delete trio replicates the
models.py pattern into instance/printedpartsimages/ with a public GET.
PrintedItemDetail follows the unified detail skeleton (hero photo,
info list, transaction history table); PrintedItemForm covers
create/edit plus photo management on edit.
GET /items (paginated, search across code/name/description/bin,
lowstock filter) and GET /items/<id> with recent transactions, both
open reads. printedpartsApi client, router file repointed at the
renamed views, PrintedItemsList with image thumbs and a red/green
quantity badge against the per-item threshold. Nav entry '3D Parts'
with a new 'box' Lucide icon mapping (the sidebar renders nothing for
unknown icon names - lab gotcha).
PrintedItem (catalog: code, name, image, cached quantityonhand,
per-item threshold, bin) and PrintedItemTransaction (the ledger:
signed quantity change attributed to a badge-resolved employee).
Both registered in PLUGIN_TABLE_OWNERS; 0001 is a post-cutover real
baseline. The migration-guard test learns the new expected head.
Routes are a placeholder ping until the next stage - the scaffold's
list route imported the deleted scaffold model, which surfaces as an
empty 'Migration error' because the alembic env imports the models
package.
flask plugin new output, minus the scaffold's AssetType seeding:
printed parts are quantity-based consumables, not ADR-001 assets.
on_install seeds the three plugin settings instead. Manifest pins
core >=0.11.0, depends on employees (badge name resolution), ships
disabled until a site opts in.
State that this is a bundled plugin whose frontend and three small core
edits land in this repo, list the three deliberate divergences from the
scaffold before the learner hits them, suggest a per-milestone solution
branch for instructors, and point out the earliest visible win (wire
the bare list page as soon as the GET endpoint works).
The kiosk take endpoint is the product's first unauthenticated
mutation; spell out the acceptance criteria (decrement-only, badge
attributed, bounded, physically rate-limited) so future open-write
endpoints meet the same bar. The dashboard-widget milestone is marked
optional: get_dashboard_widgets predates the ADR-010 data-only
renderers and needs a core component to render.
Design for a 3D-printed-parts storefront: item catalog with images and
quantity on hand, a transaction ledger attributing every take/restock/
adjust to a badge-scanned employee, an unauthenticated touch kiosk
(scan bin barcode, scan badge, keypad quantity), 1x0.5in CODE128 bin
labels, and stock/consumption/by-person reports.
The lab guide walks a developer through building it in seven
checkpointed milestones, reusing the USB badge contract, the
measuringtools migration baseline, the models-image upload trio, and
the open kiosk-endpoint precedents.
The X-Forwarded-For rewrite rule alone is not enough: waitress 2+
strips forwarded headers from untrusted proxies by default, so the app
still saw 127.0.0.1 with the rule active. Trust the loopback proxy and
consume x-forwarded-for on the waitress command line; waitress then
rewrites remote_addr to the real client. Runbook gains the
allowedServerVariables unlock (500.52) and both troubleshooting rows.
The form-control and notification-tint blocks applied on the OS
prefers-color-scheme alone, so a machine in OS dark mode leaked dark
widget styling into the app's explicit light theme - dropdown options
rendered near-black on black. The theme store always stamps data-theme
at startup, so scope these rules to [data-theme=dark]. Dropdown options
in dark mode use the solid card background instead of the text color.
Loader: bool() on pymysql bit(1) bytes is always true - isinstallable
and isshopfloor imported as 1 for every row; route through _truthy_bit.
The employee source DB is now optional (shopdb-only imports).
Frontend: under a subpath mount the 401 interceptor stored the browser
path (mount base included) as the login redirect and the router applied
its base again (/ops/ops). New stripBase() keeps redirects base-free.
tools/export-github.sh automates the publication flow: prune + scrub +
commit into ~/projects/shopdb-flask-pub and emit a transfer bundle.
Overview and plugin list cover all twelve plugins (geenforce, network
subnets, measuring tools, warranty, USB); naming examples use living
tables/columns instead of retired pctypes/isvnc; API params match the
implementation (perpage, dir, assettype); import section points at the
IMPORT-API surface and the wjf reference loader. DATA_MIGRATION_GUIDE
is now a pointer stub (its direct-DB approach is superseded).
Import-surface docs and docstrings describe the automation as a
migration script; status-doc references in CHANGELOG/ADR-009/ROADMAP
point at repo files. Screenshot/verify tools write to /tmp/shopdb-shots
(created on import) instead of a machine-specific directory.
The app can run as an IIS Application under an existing site
(e.g. https://host/ops/) instead of its own site + port:
- frontend: vite base via VITE_BASE_PATH; router history, axios
baseURL, and root-absolute asset/route paths resolve through
utils/basePath.js withBase()
- backend: MOUNT_PATH (env or .env) wraps the app in a WSGI
middleware that shifts the prefix into SCRIPT_NAME, so one knob
serves API + SPA under the mount
- docs: INSTALL-WINDOWS-IIS.md section 7b runbook + troubleshooting
rows; DEPLOY-WINDOWS-IIS.md pointer; commented examples in
deploy/windows/web.config and .env.example
Root deployment unchanged (MOUNT_PATH unset, base '/'). Also folds
two stray root-absolute callers into the shared plumbing
(MachineForm relationship-types fetch, reports CSV window.open).
Replace the free-text/datalist vendor field with a plain select populated from
/api/vendors, so a warranty vendor is always one of the site's known vendors. An
existing warranty's vendor is kept selectable even if it is absent from the
catalog, so editing never blanks it.
- Vendor is now a datalist combobox seeded from the site vendor catalog
(/api/vendors), so users pick a known vendor instead of retyping it, while
still allowing free text. Kept as a string, not an FK, to keep the warranty
plugin decoupled from core.
- Renamed the "Provider" select to "Lookup source" with helper text. It was
confusingly synonymous with Vendor; it actually means where coverage data
comes from (manual entry vs a maker's warranty API that supports Refresh).
Classic ASP keeps retired machines in the machines table as history (isactive=0);
every other stage already filters isactive=1, but the assets hub, metrology,
locations and the verify count read machines unfiltered, so ~240 retired units
(incl. all G-prefix hostnames and 61 dead metrology PCs that each synthesized a
phantom measuring tool) landed as live assets. Add the isactive=1 filter to
those four queries. Downstream stages resolve via the id crosswalk, so warranties
/comms/relationships/installs for retired machines now drop automatically.
- Perf: list_warranties did a db.session.get(Asset) per link per warranty
(~1.8s for the full list). Eager-load the links and batch-fetch every linked
asset in one query -> ~0.19s.
- Filters: the "Status" label wrapped its select onto a second line, so the
dropdown sat above the search box; keep the label inline so they align.
- Covers: each asset chip now shows the asset name (often the hostname/alias) on
hover, keeping the machine number as the label.
A freshly-imported DB left setup_complete unset, so the admin was bounced into
the first-run wizard even though the instance is fully populated. The harness now
sets setup_complete=true (it already mints the admin), so an imported instance
goes straight to the app.
Routing by pctype wrongly swept ~105 metrology PCs (CMM/Genspect/Keyence/Wax)
into measuringtools - but a PC that drives an instrument is still a computer;
the physical CMM/gauge is the tool. Dropped the pctype override: those PCs now
import as computers (measuringtools drops to the 48 real instrument rows).
Classic has no separate tool row for a metrology PC, so they'd be orphaned. New
metrology stage synthesizes a measuring-tool asset per metrology PC (typed by
its pctype: CMM / Form Tracer / Vision System / Genspect) and a Controls
relationship PC -> tool, mirroring what the runtime collector does.
Result: computers 663->751, measuringtools = 48 real + 88 synthesized (each
linked to its controlling PC), 88 new Controls relationships, no orphans.
808->966 tests (+ the migrations-mysql CI job), 11->12 bundled plugins (add
geenforce), Alembic head 7d24->7d25 (32 migrations; env.py sql_mode note for
strict MySQL 8), date to 2026-07-13, and a legacy-import pointer
(IMPORT-API/ADOPTION/PILOT-DEPLOY + the WJ reference loader).
.actions sets display:inline-flex (correct for a button container), but it was
applied directly to <td class="actions">, pulling the cell out of the table row
box so its bottom border rendered ~1px off from the other columns. Override
td.actions back to display:table-cell and space multiple buttons with a margin
instead of the flex gap.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Row-click: the whole table row now navigates to the item detail (machines, PCs,
printers, network devices, measuring tools, applications), matching the Networks
list. The actions cell is @click.stop so View/Edit/Delete still work
independently; a shared .clickable-row style gives the cursor + hover.
Network hub: drop the VLANs tab - a subnet belongs to a VLAN (each Networks row
already shows its VLAN) so a sibling tab was redundant; VLAN naming stays in
Settings. Hub is now Devices | Networks.
Verified: row-click navigates on machines/pcs/network; hub shows two tabs.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replaces the two flat "Network Devices" + "View Networks" nav entries with a
single "Network" entry opening a tabbed hub: Devices | Networks | VLANs
(NetworkHub renders the existing device list, the subnet browse, and the VLAN
list; VLANs is now reachable outside Settings). /network -> hub; /networks
redirects to the Networks tab; subnet detail stays at /networks/:id.
Subnet "Devices on this network" now matches ANY asset whose primary IP falls in
the CIDR (PCs, printers, machines, measuring tools - not just network devices),
computed on the core Communication + Asset tables; each row links to its typed
detail (extension id resolved lazily/guarded per plugin). Fixes the empty list -
printers and PCs carry IPs and now appear (e.g. 35 devices on 10.80.92.0/24).
Also: subnet-browse search uses the standard form-control styling; dropped the
redundant per-tab page header.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Subnets previously lived only under Settings, easily confused with the Network
Devices asset list. Add a front-facing browse + detail:
- Nav: rename "Network" -> "Network Devices"; add "View Networks" (subnets), both
under Assets (network plugin get_navigation_items; frontend fallback matched).
- /networks (SubnetsBrowse): all subnets with name / CIDR / type / VLAN / notes,
searchable, row-click to detail.
- /networks/:id (SubnetDetail): the subnet (CIDR, network address, type, VLAN,
gateway, notes) plus the network devices whose primary IP falls inside its
CIDR - get_subnet now computes that membership in Python (a device's IP lives
in a Communication row, so it is not a plain SQL join).
Verified on the import DB: 37 networks list (real WJ subnets), detail renders.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
.relationships-section had card styling but lacked the margin-bottom +
break-inside:avoid that .section-card has, so it sat flush against the Notes
card below it and looked like one merged card (and could split across a multicol
break). Add both to match section-card. Affects every asset detail page.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
USB Checkout History rendered every record unbounded (long table for heavy
users), inconsistent with the recognitions list right above it. Add the same
limit (10) + "Show N more" / "Show less" toggle recognitions use. Client-side
only; no API change (per-user history is small at current scale).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Classic ShopDB listed an application's related KB articles on its page; the flask
app detail didn't, even though KB articles carry an appid FK. get_application now
returns a knowledgebase list (KB rows linked by appid; lazy + plugin-guarded so
core stays decoupled), and ApplicationDetail.vue renders a Knowledge Base section
linking each article.
Verified: an app returns its linked KB (e.g. 77 articles) and the section renders.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The applications stage dropped applicationlink and documentationpath, so the app
detail's "Launch Application" and "Documentation" links were always empty. Map
both from the classic applications table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
_search_employees always queried the external HR database (employee_connection),
so a site running the selfhosted employee directory (the app-owned
directoryemployees table) got zero employee results - searching an SSO or name
found nothing, and there was no way to reach the employee profile. Made it
mode-aware via the employee_directory_mode setting: selfhosted -> query the
DirectoryEmployee table (lazy, plugin-guarded); external -> the HR DB as before.
Verified: SSO 210009518 -> Jeff Pierce -> /employees/210009518; name "Pierce"
-> Pierce Cox, Andy Pierce, Jeff Pierce.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A recognition/training notification for many people prefixed the calendar title
with the entire roster ("Name1, Name2, ... , Name20: description"), burying the
description. to_calendar_event now shows "First Person +N" when more than one
person is listed; single-person titles are unchanged and the detail popup still
shows the full employeename. Fixes the cluttered month grid, especially after
the loader began resolving employee SSOs to names.
31 notification/calendar tests pass; naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Notifications imported with only employeesso, leaving employeename null - the
model displays "employeename or employeesso", so recognition/training cards
showed a bare SSO instead of a name. Build an SSO -> "First Last" map from the
employee source and populate employeename (comma-separated SSOs -> joined
names). SSOs not in the directory (former/non-WJF) stay null and fall back to
the SSO, as before.
Verified on the import DB: 261 notifications re-imported, names resolved
(Brandon Saltz, Jon Kolkmann, ...).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Found by test-deploying on a Windows + MySQL 8.0 VM: migration 7a01 seeds the
canonical relationship types with a raw INSERT that omits the NOT-NULL
createddate/modifieddate columns (the ORM supplies those via Python defaults at
runtime, but a raw migration INSERT does not). MySQL 5.x's lax default sql_mode
accepted it; strict MySQL 8 rejects it with 1364 "Field 'createddate' doesn't
have a default value", so a fresh `flask db upgrade` died at 7a01. Dev runs
MySQL 5.6, so this never surfaced locally.
migrations/env.py now sets the migration session sql_mode to
NO_ENGINE_SUBSTITUTION (dropping STRICT_TRANS_TABLES) for the migration run
only - the app's own runtime connections keep their mode. Makes the whole chain
portable across MySQL versions. Guarded for non-MySQL (sqlite tests).
68 migration/smoke tests pass; fresh upgrade to head verified on MySQL.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
docs/PILOT-DEPLOY.md ties the generic per-site deploy (DEPLOY.md) to the legacy
import: pre-flight, stand up an empty instance, enable all plugins (incl usb),
load the three classic dumps into scratch DBs, run the WJ loader against the
pilot DB, verify (row-count audit + UI spot-check checklist), a parallel-run
window, cutover, rollback, and post-cutover (backups, photos, GE-Enforce).
Includes the expected import magnitudes from the dev run.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Follow-up to the mapper retirement (the new docs missed the prior commit's
staging). Adds docs/IMPORT-ADOPTION.md (two-layer import story + stage/crosswalk
guidance), scripts/migration/README.md (dir superseded, points at the API +
loader), and updates the WJ loader README to complete status.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Cleanup after the reference loader (scripts/site_imports/wjf/) proved out.
Removed the drifted direct-SQL migrators - migrate_assets/communications/
notifications/usb.py, run_migration.py, verify_migration.py, and
scripts/import_from_mysql.py. They targeted a nonexistent equipment table, the
retired Machine model, and columns that no longer exist; nothing imported them.
scripts/migration/README.md now points at the import API + the site loader.
Kept the one-time SQL fixups (fix_legacy_schema.sql, one-offs/).
Added docs/IMPORT-ADOPTION.md: the two-layer import story (stable IMPORT-API
contract + per-site loader), stage-ordering + crosswalk guidance, the
agent-assisted mapping path, and what the WJ loader demonstrates. Updated the
loader README to complete status (all 15 stages, final counts).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
printers: printers come from the printers TABLE (not the machines hub), so a
dedicated stage - assetnumber synthesized PRN-{printerid}, IP folded via the
create route, host machineid resolved to a location when it is a LocationOnly
row. Skips inactive. 50 of 56 imported.
relationships: a "Controlled By" edge now flips to the forward Controls
direction (source/target swapped, mapped to the Controls type) instead of
importing a redundant inverse type.
Final fresh full run: 983 assets (933 machines-hub + 50 printers), zero endpoint
errors, all 15 stages.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Completes the loader end to end. Verified on a fresh scratch target, zero
endpoint errors:
- locations: the 24 islocationonly rows -> core Locations (crosswalk machineid
-> locationid).
- relationships: 93 active edges imported (206 of 299 dropped because an
endpoint became a Location / was skipped / dedup-lost); types folded onto the
seeded canonical set; dedup on (source,target,type).
- subnets: 37 (full CIDR reconstructed as INET_NTOA(ipstart)+suffix; VLANs
lookup-or-create by number; 3 duplicate CIDRs first-wins-skipped).
- usb: 18 cmmc devices + 232 check-in/out events, paired with per-device open
state so unpaired log rows do not 400. Needs the usb plugin enabled + usb
directory mode selfhosted.
- verify: source-vs-target row-count audit (assets 1167->933 by the skip rules,
applications 121=121, employees 415=415, KB 342->341).
Full pipeline default runs all 14 stages in order. NOTE: the import target needs
every bundled plugin enabled (usb ships disabled in this dev registry - enable
it before importing).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Of the 158 machinetypeid=1 rows, only 24 carry the islocationonly bit (real
named areas: DT Office, IT Closet, Materials, ...). The other 134 are active,
modelled shop machines just left untyped - routing all 158 to Locations dropped
those 134 real assets. Route on the bit instead; the 134 untyped rows import as
machines with a null subtype (machinetypeid=1 is not a real machine subtype, so
catalog skips seeding one).
Also process asset routes in richness order (computer > measuringtool > network
> machine) so on a duplicate machinenumber the PC - which carries installs + IP
a bare untyped machine does not - wins first-come.
Result on the scratch target: 933 assets (computer 663, machine 76, network 58,
measuringtool 136), 24 locations (was mis-routing 158), installs 850 (was 653 -
PCs no longer lose their numbers to bare machines), warranties 464, comms 461.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
All consume the machineid->assetid crosswalk from the assets hub. Verified
against the scratch target, zero endpoint errors:
- communications: 435 primary IPs folded onto assets. No bulk endpoint exists,
so this is the plan's documented direct-ORM gap (reads the source
communications table where comstypeid=1 AND isprimary=1, not machines.ipaddress1
which is empty).
- applications: supportteams 45, applications 121 (colliding names dedup via the
unique-appname 409-resolve), appversions 47, installs 653 (machineid ->
assetid -> computerid; only computer assets take installs).
- warranties: 424 linked, vendor hardcoded Dell (source has none).
- notifications: types 6, notifications 261 (2099 sentinel endtime clamped).
- knowledgebase: 341 (appid resolved through the applications name map).
Inactive rows skipped everywhere per the decisions. Remaining loader stages:
locations (the 158 LocationOnly rows), relationships (301 active edges),
subnets/VLANs, usb (cmmc pairing), verify.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds the keystone stages to the reference loader.
catalog: seeds modeltypes (from classic machinetypes + category), the per-plugin
asset subtypes routed by machinetype (machines/network/measuringtool types),
computer subtypes (from pctype), the 5-row controllertypes vendor/model split,
and the models catalog - each with a persisted legacy->new crosswalk. Verified:
modeltypes 31, computertypes 12, models 118.
assets (the hub): fans classic machines out to the right endpoint by
machinetypeid (+ the pctype metrology override), applying the resolved
decisions - assetnumber = machinenumber else hostname, skip 9999, skip duplicate
machinenumbers, LocationOnly/printer/USB routed out. Persists the machineid ->
assetid crosswalk every downstream stage needs. Verified against a fresh scratch
target: 884 assets (computer 623, machine 68, network 58, measuringtool 135),
zero endpoint errors, idempotent re-run (stays 884). Skips: location 158, dup 69,
other 53, 9999 1.
Harness now runs each plugin's idempotent on_install so the AssetType rows exist
(a DB built with plugin upgrade-all instead of a fresh install lacks them, and
the create routes 500 without them). 409-resolve lookups page through per_page.
Remaining stages: communications (primary IP fold - source is the communications
table, not machines.ipaddress1), applications/installs, warranties, notifications,
KB, subnets/VLANs, usb, verify.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Layer 2 of the import design (see the memory + scratchpad/IMPORT-PLAN.md): a
SITE-SPECIFIC reference loader that maps WJ's classic-ASP schema onto the
maintained, schema-agnostic IMPORT-API contract. Other sites copy the pattern
against their own source DB; nobody runs this loader as-is.
Harness (scripts/site_imports/wjf/harness.py): builds the app against the
current DATABASE_URL (point it at a throwaway import DB), mints an unscoped
admin PAT in-process, and drives the real import endpoints through the app test
client with Authorization: Bearer + X-Import-Mode - exercising the same
routes/authz/validation an HTTP client would, no running server needed.
Read-only pymysql access to the three scratch source DBs; legacy-id -> new-id
crosswalks persist to JSON so a crashed run resumes and later stages resolve FKs.
Stages implemented + verified idempotent against a fresh scratch target
(shopdb_flask_import): reference (vendors 46, businessunits 13, operatingsystems
11) and employees (directory 415, re-run updated-not-duplicated). Remaining
stages (models, applications, assets hub + crosswalk, dependents, network, usb,
verify) are stubbed with the same shape; README documents the adoption playbook.
idmap.json is generated state (gitignored).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
POST /api/applications/{id}/versions now calls apply_import_timestamps so an
imported version's original dateadded/releasedate survives (was skipped, unlike
the app + install endpoints). No-op outside import mode.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The legacy-import surface (docs/IMPORT-API.md) is the schema-agnostic contract
every adopting site targets; these close two gaps found while mapping the WJ
classic import.
Network device IP: POST/PUT /api/network now accept an `ipaddress` and
materialize a primary Communication (mirroring the printer route), and GET
(list + detail + create/update result) surface it. Previously a network
device's IP - which lives in the communications table, not on the extension -
had no HTTP import path at all.
communicationtypes seed: `flask seed reference-data` now seeds the eight
canonical communication types (IP/Serial/Network_Interface/USB/Parallel/VNC/
FTP/DNC), which IMPORT-API.md already documents as a prerequisite. The IP type
must exist before any asset import so printer/network routes can attach an IP.
There is no CRUD endpoint for these, so seeding is the only path.
Tests: network create/update IP upsert + GET surfacing + seed creates IP type.
204 targeted tests pass; naming + pyflakes green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
From the database review (verdict: sound-with-minor-issues). Applies the
actionable findings.
Redundant indexes: five non-unique secondary indexes duplicated a named idx_*
or a unique index on the same column - ix_communications_assetid,
ix_computers_hostname, ix_networkdevices_hostname, ix_printers_hostname (each
shadowing an idx_*), and idx_usb_serial (shadowing the serialnumber unique
index). Removed the redundant index source from the models (column index=True /
the extra db.Index) and added core migration 7d25 dropping the live duplicates.
The unique ix_*_assetid indexes are kept (they enforce assetid uniqueness).
Dead column: usbcheckouts.machineid was a NOT NULL soft-ref to the retired
machines table storing sentinel 0 (ADR-001). Dropped from the model + the
machineid=0 literal in selfhosted checkout; usb plugin migration 0002 drops it
live (downgrade restores it default 0).
Index: notifications.businessunitid (filtered by the shopfloor feed) was
unindexed; added index=True + notifications migration 0002.
CI: new migrations-mysql job proves the real multi-site deploy path - fresh
`flask db upgrade` + per-plugin install on utf8mb4 MySQL from empty, asserting
table count + charset and a clean second-run no-op. The pytest suite only
exercises SQLite create_all(), so a regression in the Alembic chain on MySQL
would otherwise ship undetected.
Verified: fresh core upgrade on a scratch utf8mb4 MySQL builds clean + no-op on
rerun (redundant indexes absent, unique assetid kept); plugin migrations applied
+ verified on the dev DB (machineid gone, bu index present). 953 backend tests
pass; naming + pyflakes green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
DB review found four DateTime columns defaulting to db.func.now() (MySQL
session-timezone wall clock) while the rest of the schema stores naive UTC, so
one schema mixed two clocks and to_dict() labelled the local values UTC with a
'Z' suffix. Switch application.dateadded, computers.installeddate,
knowledgebase.lastupdated (default + onupdate), and slides.uploadeddate to the
module-level naive-UTC _utcnow callable already used elsewhere (apitoken.py).
ORM-side default only - no column-type change, no data migration; affects
new/updated rows.
Targeted tests pass (154); naming + pyflakes green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Naming convention (LOCKED): rename the ManifestEditor simulate state sim ->
simulateInputs / simResult -> simulateResult (+ .sim-result CSS class) - 'sim'
was banned standalone shorthand. Rename AssetRelationships props assetId ->
assetid and machineNumber -> machinenumber so a prop holding a DB field value
mirrors it verbatim; updated the five detail-page call sites (:assetid=).
CSS variables: SearchResults per-domain badge palette moved into CSS variables
on the container; the duplicated prefers-color-scheme dark block collapses to a
single set of variable overrides instead of restating all ten selectors.
frontend build green; vitest 49 pass; naming green; search badges + detail
relationships verified rendering with no console errors.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Review flagged ROADMAP as six contract versions stale. Set opening version to
0.11.0; mark phase 6 (multi-site distribution) done and name the real last
milestone (legacy import + prod pilot); drop the three completed items
(per-plugin Alembic chains per ADR-008, local font bundling - Inter is bundled
via @fontsource, measuringtools plugin built); reframe the frontend item to the
part that actually remains (external plugin UI packaging - hooks + gating
already shipped via ADR-009/010); add ADR-007..012 to the decision-log pointers.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fills the review's highest-value coverage gaps in the framework's own core
feature.
New tests/test_plugin_lifecycle.py: manager enable/disable dependency guards
(beta depends on alpha - enable-beta-first refused, disable-alpha-while-beta-on
refused) via synthetic plugins; enable seeds a plugin's RBAC permissions
idempotently; registry disable-survives-reload, corrupt-file recovery, and the
equipment->machines rename migration; `flask seed settings` idempotency.
test_plugin_contract.py: BUNDLED_PLUGINS now covers all bundled plugins incl.
geenforce, measuringtools, warranty (was 9, contradicting the CLAUDE.md "all
bundled satisfy the contract" claim); the structural checks now run against them.
test_authz.py: account lockout auto-unlock path (expired lockeduntil -> correct
password logs in and clears the lock state), previously untested.
All new tests pass; naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Settings exposure (review medium): GET /api/settings and /api/settings/<key>
now return the full table only to an authenticated principal. Unauthenticated
callers (kiosk dashboards, print pages, login screen, setup router) get just a
public allowlist - categories branding + map plus a named set (site_base_url,
facility_name, printer_hostname_template, contact_email_domain,
servicenow_enabled, setup_complete). A non-public single-key GET returns 404 so
existence is not confirmed. Secrets stay masked in both cases. Closes the
unauthenticated enumeration of smtp_host / employee_db_host / zabbix_url /
servicenow URLs. Allowlist mirrors the keys siteSettings.js + mapConfig.js +
setupState.js read before login.
audit.view (review low): the three audit-read routes (list, entity-history,
stats) were jwt_required only despite a defined-but-unwired audit.view
permission; now gated by it (seeded to admin), so a role-less member or unscoped
PAT can no longer read the cross-user audit trail.
flask seed test-user (review low): refuses outside DEBUG/TESTING - it creates
the well-known admin/admin123; production sites use `flask seed admin`.
Tests: unauthenticated allowlist + authed-full-masked + private-key-404, and
member-403 / admin-200 on audit routes. 336 authz tests pass; naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Plugins list now carries a displayname (manifest display_name, else the
machine name title-cased). Adds display_name to the four whose title-case
was wrong: GE-Enforce, USB, Measuring Tools, Knowledge Base. The setup
wizard Features step and Settings > Plugins render it, so "Geenforce"/"Usb"
are gone.
Finish step shows a pointer when GE-Enforce is enabled: it still needs a
scoped service token (Settings > API Tokens) and a share export root
(GE-Enforce page) before the fleet uses it - operational config the wizard
does not collect.
frontend build green; naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Per-method detection hint: a plain-language line explains what "already
correct" means for the selected detection method (Registry/File/FileVersion/
Hash/MarkerFile/ValueMatches/pnputil/Always), updating live as the author
picks one. Lives in entryForm.js (DETECTION_METHOD_HINTS + detectionMethodHint)
with 4 new vitest cases; the editor renders it under the dropdown.
Also: relative-path hint on Installer/Source (path under the scope payload
folder or an inline payload), an InUseCheck behavior hint, and refresh the
entryForm.js header now that the editor imports these helpers directly.
vitest 49 pass; frontend build green; naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add test_geenforce_ddl_parity to lock the manifest models against their
Alembic baseline (catches model/migration drift for a chain that is still
amendable pre-deploy).
Retire the "Collector PC Types" settings card: GE-Enforce scope
computertypeid supersedes the pctypemap editor UI (ADR-012). The collector
still reads pctype_mapping(), so the backend map stays; only the editor
surface is removed, with a deprecation note in pctypemap.py.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fleet-install compliance for app-linked manifest entries: new service
compliance_for_scope + GET /geenforce/scopes/<id>/compliance count active
ComputerInstalledApp rows by curated appid (null-safe when computers plugin
absent). ManifestEditor gains a compliance panel. Curated appid stays shopdb
metadata and never enters manifest JSON, so behavioral parity is unaffected.
Inline manifest payloads: store_inline_payload (sha256, 1MB cap,
payloadsource='inline') + POST/GET /geenforce/entries/<id>/payload; editor
gains an upload control. Entry payload metadata surfaced in _entry_payload.
Frontend test harness: extract the editor's entry-form logic into pure
entryForm.js (buildEntryPayload, describeEntry, availableEntryTypes, scope
gates, ...) and cover it with 45 vitest tests. ManifestEditor now imports
those helpers, so the tests exercise the shipped code path (no duplication).
908 backend tests pass; vitest 45 pass; frontend build green; naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The honest replacement for the backed-out auto-seeding: instead of scraping
manifest labels into duplicate Application rows, an entry can be LINKED to an
existing catalog Application, cross-referencing what shopdb already tracks.
- Model: manifestentries.appid (nullable soft ref to core applications; in the
0001 baseline). It is shopdb METADATA, deliberately NOT a manifest field - it
never appears in the rendered manifest JSON, so enforcement + parity are
unaffected (test asserts it stays out of the preview manifest).
- API: _entry_payload returns appid + resolved appname; create/update accept an
optional appid (validated, unknown id ignored, null unlinks) via _apply_app_link;
GET /geenforce/applications is the picker source (id + name).
- Editor: a "Tracked application (optional)" select in the entry modal, and the
entry summary line notes the linked app ("...; tracked: eDNC").
- Foundation for a future desired-vs-observed compliance view.
889 tests green (incl. the link test + parity/migration unaffected); build +
naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Found by running the kit under pwsh 7 against the live API: Invoke-WebRequest
returns header values as string ARRAYS in PS7 (scalars in Windows PowerShell
5.1), so X-Manifest-Version came back as @('1') and [int] on it threw - report
build failed. The target scheduled task runs 5.1 (works), but the kit must be
robust under PS7 too (target preinstalls PowerShell 7). Coerce ETag and
X-Manifest-Version with @(...)[0], a clean scalar in both.
Validated end to end on Linux pwsh 7.6.3 against the dev API: fetch (200) ->
cache-304 -> report sent -> landed received=true/status=ok. All 4 client scripts
parse clean; PSScriptAnalyzer shows only cosmetic warnings (Write-Host in a CLI,
intentional log-guard catch).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes the "how do sites actually deploy GE-Enforce" gap (esp. OOBE-ppkg sites
without a PXE/WinPE step). Site-neutral + imaging-path independent.
- plugins/geenforce/client/Install-GEEnforce.ps1: a bootstrap that writes the
PC's identity (C:\Enrollment\pc-type.txt is what determines the PC type; plus
machine-number/cmm version/cmm id/site-config as needed), sets the shopdb
BaseUrl + token in HKLM:\SOFTWARE\GE\ShopDB, deploys the client kit, optionally
copies the engine from -EngineSource, and registers the SYSTEM scheduled task
(at logon + every N min). Idempotent; fails loud (installer, not the fail-safe
runtime). Engine is REFERENCED not vendored - it belongs to the GE-Enforce
framework; the script warns if absent but still labels the PC.
- docs/GE-ENFORCE-DEPLOY.md: the deploy contract - the three things a PC needs
(client, identity, credential), the identity table (what determines PC type,
no auto-detection - the provisioner supplies it; shopdb cannot set it at
imaging), and how to invoke per path (PXE step, OOBE ppkg via
ProvisioningCommands, Intune, manual), the engine boundary, and verification.
- Cross-linked from docs/GE-ENFORCE.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Phase-aware editing: a preinstall scope now only offers MSI/EXE types and
Registry/File detection (the preinstall runner silently skips the rest), and
the preinstall flags show only for a preinstall scope - so an author cannot
pick an option that would do nothing.
- Per-scope summary line ('Installs PC-DMIS 2016, ...; 4 entries; runs after
common') under the scope header.
- Entries table: dropped the redundant Detection + Filters columns (the
plain-English entry line already conveys them), fixed-layout with sized button
columns and stacked Up/Down - no more horizontal scroll.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verified claim-by-claim against the engine/dispatcher/preinstall runner/manifests.
Corrections (3 were ship-blocking):
- Timeline (ship-blocking): identity files (pc-type/machine-number/cmm version/
site-config) are written in WinPE at the PXE menu BEFORE the image boots, not
during a post-imaging 'enrollment' step; preinstall already reads them. Added a
step [0]; enrollment is now only Intune + Azure DSC credential provisioning.
- _CmmVersion (ship-blocking): a CMM bay with NO resolved version gets ALL
PC-DMIS versions (legacy install-all), not none.
- machine-number 9999 (ship-blocking): the enforcement engine does not
special-case 9999; it is a placeholder that won't match real bay gates (the
9999-skip is status-write-back only).
- Preinstall runner implements only MSI/EXE + Registry/File detection, not the
full matrix (that is runtime-only).
- Runtime processes up to three scopes: common, type, then optional type-subtype.
- pc-subtype.txt is legacy (no longer written at imaging since 2026-05-04).
- The collector ComputerType mapping lives at Settings > Collector PC Types, not
the geenforce scope (scope computertypeid is a local reference field).
- FileVersion is a raw string compare; 4-part is convention, not engine-enforced.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Each manifest entry now shows a one-line intent summary under its name
(e.g. 'Installs eDNC; reinstalls if not detected; 2 PC types') instead of only
the raw Type/Detection/Filters columns - turns jargon into what it actually does.
- A collapsible 'How this works' panel at the top gives the mental model in four
sentences (desired state + self-heal, entries + detection, targeting, doc link).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
docs/GE-ENFORCE.md - operator-facing guide grounded in the real engine +
manifests (Fable-verified analysis): how GE-Enforce works (preinstall vs runtime
phases, the enforce loop, manifest scopes/entries, self-heal detection, gates,
enrollment), WHEN it installs/takes over in the imaging timeline (preinstall at
imaging -> GE-Enforce laid down -> enrollment provisions creds -> runtime
enforcement from first logon), how the shopdb plugin manages it (Manifests
authoring + contextual targeting + simulate + publish/rollback + Export to Share
Milestone 1, Enforcement Reports), day-to-day IT tasks, and a reference index.
Complements GE-ENFORCE-CLIENT.md (client contract) and the proposal (plan).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the generic field-dump editor with the Fable-coordinated redesign
(stock .section-card / .settings-grid / .setting-row / .table-container /
badge / btn primitives, opaque global .modal on --bg-card-solid, proper
modal-header/body/footer), then layer the manifest semantics on top:
- CONTEXTUAL targeting: an entry only shows the gates its scope actually uses,
instead of every gate on every entry. Data-driven + scope-aware -
common/preinstall (fleet-wide) show PC types; CMM shows the version gate;
a scope whose entries use machine numbers (collections) shows those; hostnames
show when used. A short explainer states why (a per-type manifest already runs
only on its own type), and "Show all targeting options" reveals everything.
Hidden gates keep their value on save (no data loss).
- Preserves all functionality: type-switched payload blocks, detection reveal,
structured InUseCheck rows (name/ExePath/timeout), LogFile, preinstall
checkboxes, simulate, publish/versions/rollback, export. Entry modal widened
to min(1100px,96vw).
EnforcementReports adopts the themed redesign (filters, card > table-container,
badge-mapped statuses, global modal) with the error handling kept.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A deep Fable review of the real manifest corpus (READ-ONLY reference) showed the
manifests are an ENFORCEMENT PROGRAM, not an application inventory, and that
auto-seeding the Applications catalog from entry Type + Name was wrong:
- The catalog ALREADY tracks these apps from the classic-shopdb migration, with
version histories (PC - DMIS, UDC x11 versions, eMX / eDNC, CLM, CSF, Oracle
Database, FormTracePak). Seeding from manifest labels created DUPLICATES under
different names (PC-DMIS 2016 vs PC - DMIS; eDNC (bundles NTLARS) vs eMX / eDNC;
OpenText HostExplorer ShopFloor vs CSF). It also misclassified config drops
(eMxInfo.txt) as apps and could never match a PC's reported ARP name.
So the seed-applications command + service are removed. Properly linking
manifest entries to the EXISTING catalog is a curated feature, not label-scraping.
Two REAL bugs the review found are fixed and kept:
- Report status (R4): every healthy cycle runs Always/no-detection scripts the
engine counts as "installed", so keying self-heal off installed>0 marked the
common scope selfhealed forever and made 'ok' unreachable. Status now derives
from explicit per-entry self-heal flags only; the stored flag no longer infers
from action=='installed'; the client kit doc reflects it.
- PCTypesStrict (R5): the runtime engine has no strict handling (preinstall
runner only). filters.matches_pctype now applies strict only when phase ==
'preinstall'; simulate + parity thread the scope phase through; the strict test
uses a preinstall scope.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
seed-applications: a flask geenforce seed-applications command + service that
reads the imaging-PC-type manifests and creates a core Application for every
installer entry (MSI/EXE/CMD/BAT), so shopdb tracks what GE-Enforce actually
deploys. Idempotent, deduped by appname; File/Registry/PS1/INF config entries
are skipped. Run against the West Jefferson reference: 27 apps tracked (PC-DMIS
2016/2019/2026, eDNC, Oracle Client, Adobe Reader, HostExplorer, the VC++ redist
matrix, Keyence VR-6000, PowerShell, Display Kiosk, ...). 2 tests.
Editor: the CMM version gate (_CmmVersion) now only shows for CMM scopes - it is
metrology-specific, so a printer/common entry form no longer carries the
irrelevant field.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
--bg-card is translucent (rgba .4) in dark mode - a glass effect for cards on
the page, but a floating modal rendered over content showed the page through it.
Modals now use --bg-card-solid (the opaque surface the shared Modal.vue uses),
with a border + shadow, and are wider (entry editor min(920px,94vw)) so the full
field set fits.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The earlier move copied instead of moving, leaving unreachable duplicates under
views/settings/. The routed copies live in views/geenforce/.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
GE-Enforce is a large operational surface (manifest authoring + fleet
compliance), not a setting, and it was squished in the settings two-pane shell.
Promote it to a dedicated full-width top-level section:
- New sidebar entry "GE-Enforce" (plugin get_navigation_items, shield icon,
auto-gated to the enabled plugin) instead of two Settings > Integrations cards.
- Tabbed shell GeEnforceLayout.vue (Manifests | Enforcement Reports) with
full-width children under AppLayout, not the narrow settings rail.
- Views moved settings/ -> geenforce/ (ManifestEditor.vue, EnforcementReports.vue).
Theming + overflow fixes (the "chaotic / cut off / different inputs" report):
- Inputs/selects/textareas now match the stock settings look (border, radius,
--bg, focus color) instead of browser defaults.
- No horizontal overflow: editor grid uses minmax(0,1fr) + min-width:0 on
children, collapses to one column under 1000px; entry table and reports table
scroll inside their own overflow-x containers; detail actions wrap.
Verified at 1280px: no page overflow, detail pane + tables fit.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Consolidated fixes from a three-dimension adversarial review.
Data-loss (HIGH): the manifest entry editor stripped fields the form did not
expose, because PUT /entries is a full reset-then-apply. The form now captures
everything - InUseCheck processes as structured name/ExePath/timeout rows (not
just names), LogFile, and the three preinstall flags as checkboxes; the dead
payload-source control (never wired) is removed. New regression test proves an
edit preserves ExePath/timeout/LogFile/PreEnrollment/PCTypesStrict.
Update-entry crash (found by that regression test): replacing an entry's
one-to-one InUseCheck (unique entryid) collided with the old row mid-flush ->
IntegrityError -> 400. update_entry now frees the old InUseCheck (delete+flush)
before populate re-inserts it.
Export truncation (MEDIUM): export_scope_to_share used a plain truncating open,
so a failed/partial write left the live on-share manifest (every PC reads it)
empty. Now writes a temp file in the same dir and os.replace() atomically.
Report dedup case bug (MEDIUM, confirmed by scratch test): the iscurrent demote
matched hostname case-sensitively while the read path uses ilike, so a PC
reporting different casing left two iscurrent rows and double-counted. Demote is
now case-insensitive; regression test added.
Simulator fidelity (MEDIUM): PCTypesStrict was captured but ignored by the
filter mirror, so the simulator wrongly matched a collections-only strict entry
to a nocollections PC via the shared Standard alias group. matches_pctype now
honors PCTypesStrict (disables alias expansion); test added.
Hardening: removed the dead/unscoped GEENFORCE_API_KEY env fallback (never wired
into config; tokens are the only path); create/update entry return 400 on a
duplicate Name instead of 500; parity now asserts scope-level Version/Site; a
new test guards real-manifest field lengths against column limits (the DB-free
parity harness can't see truncation); error handling added to the previously
unguarded editor + reports API calls.
Full suite green; naming + frontend build green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Client-side integration kit for sourcing manifests from shopdb and reporting
results back. Site-neutral reference a site adapts into its GE-Enforce.ps1; the
live dispatcher and engine are NOT touched (they are read-only reference under
projects/pxe). Only the manifest JSON source moves from a share file to shopdb,
plus a result report.
- plugins/geenforce/client/ShopdbEnforceClient.psm1: Sync-ShopdbManifest (GET
with ETag -> local cache; falls back to last-known-good when shopdb is
unreachable so a PC is never left unmanaged), Compare-ShopdbShadow (behavioral
diff vs the on-share manifest), Send-ShopdbReport / New-ShopdbReport (best-
effort POST /report), Get-ShopdbConfig (BaseUrl + token from
HKLM:\SOFTWARE\GE\ShopDB).
- plugins/geenforce/client/Invoke-ShopdbEnforce.ps1: orchestrator. Fetches,
optionally shadow-compares (installs from the share, only logs the diff), runs
the unchanged engine, and reports. Fail-safe: any error exits 0.
- docs/GE-ENFORCE-CLIENT.md: the fetch + report contracts, config, cache/fail-
safe behavior, the staged shadow -> read-cutover -> payload-migration runbook,
and TLS/payload-integrity notes.
The report JSON shape matches the POST /api/geenforce/report contract already
covered by the reporting tests. Nothing here runs the live client; shadow mode
and cutover stay a site decision after Milestone 1 sign-off.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rounds out the Milestone 1 admin experience: author + publish in shopdb, push
to the share by a button, and see what the fleet actually did.
Export to share:
- GET/PUT /api/geenforce/config stores the on-share export root (Setting
geenforce_share_root); POST /scopes/<id>/export-share writes the current
published JSON to <shareroot>/<scope>/manifest.json (preinstall.json for the
preinstall phase), backing up the existing file to _meta/history first.
geenforce.publish gated. The engine and PCs are untouched - this is the safe
Milestone 1 push whose rollback is restoring the history backup.
- Editor: a share-root config row + an "Export to Share" button per scope.
- 3 tests (config roundtrip, export writes the file, second export backs up).
Fleet-compliance UI (Settings > Enforcement Reports):
- New page over GET /reports + /reports/<id>: latest report per PC with
received (applied vs latest published version), status (ok/selfhealed/failed),
and install/skip/fail counts; row detail shows per-entry outcomes with
self-heal flags, exit codes, and messages. Hostname/PC-type filters.
- ADR-010 settings card + ADR-009 plugin-gated route.
Full suite 883 green; frontend build + naming green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The imaging-PC-type manifest editor, contributed as an ADR-010 settings card
(Integrations group) and an ADR-009 plugin-gated route
(/settings/imagingpctypes, hidden when geenforce is disabled).
- Scope list: every imaging PC type with phase, entry count, and published
version (or "unpublished"). New PC Type button.
- Scope detail: ComputerType/MeasuringToolType mapping + description; Publish,
Versions (with per-version Roll Back), Preview (draft JSON), Delete.
- Entry table: ordered with Move Up/Down (the ordering contract, not drag),
Name/Type/Detection/Filters, Edit/Delete. Add Entry opens a typed modal whose
fields switch on entry Type (MSI/EXE/... vs PS1 vs File vs Registry), with a
detection block, comma-separated targeting filters, CMM version gate, payload
source, and an Advanced disclosure for the inert ApplyMode/UpdateWindow and
InUseCheck. RegValue is typed by RegType (DWord/QWord -> number).
- Simulator: "what would a PC get" - enter a machine profile, see which entries
apply and which filter excluded the rest. Verified live: CMM version 2019 ->
applies 2019 + untagged, filters 2016/2026 by _CmmVersion.
Uses the P2 admin API; JWT+admin gated. Frontend build + naming green; full
backend suite 876 green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Full HTTP admin surface behind the manifest editor (geenforce.manage for edits,
geenforce.publish for shipping):
- Scopes: POST/GET/PUT/DELETE /scopes[/<id>] (create imaging PC types, edit the
ComputerType/MeasuringToolType mapping + metadata, delete).
- Entries: POST /scopes/<id>/entries, PUT/DELETE /entries/<id>. Payloads use the
manifest Applications[] shape; populate_entry (refactored out of build_entry)
updates an entry in place, resetting omitted fields and replacing children.
- Reorder: PUT /scopes/<id>/entries/reorder enforces the ordering contract
(body must list exactly the scope's entry ids).
- Simulate: GET /scopes/<id>/simulate?pctype&subtype&hostname&machinenumber&
cmmversion returns which entries apply and which filter excluded the rest,
reusing the engine-mirror filters. The "what would this PC get" tool.
- Publish lifecycle: POST /scopes/<id>/publish (records publishedby from JWT),
GET /scopes/<id>/versions, GET .../versions/<n> (frozen manifest),
POST /scopes/<id>/rollback.
Entry type validated against ENTRY_TYPES; 8 CRUD tests. JWT+permission gated so
the authz sweep covers them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
PCs now report enforcement results back to shopdb, closing the desired-vs-observed
loop.
- POST /api/geenforce/report (geenforce.report service token): each cycle a PC
posts the published version it applied, install/skip/fail/filtered counts, and
per-entry outcomes.
- Two tables: manifestenforcementreports (latest-per-host + history: applied
version, enforcer version, counts, derived status ok/selfhealed/failed) and
manifestenforcementresults (per entry: action installed/skipped/failed,
selfhealed flag, exit code, warning/error message).
- RECEIVED: reports carry the applied version; the admin view derives
receivedlatest by comparing it to the scope's current published version, so
the fleet view shows which PCs picked up an update.
- SELF-HEAL: per-entry action captures drift correction (installed when it
should already be present) vs skipped (already good) vs failed, with messages.
- Admin reads: GET /reports (fleet compliance rollup) and GET /reports/<id>
(per-entry detail). New geenforce.report permission.
- Tables added to the (undeployed) 0001 baseline; geenforce.post_report is a
service-token endpoint so it is exempt from the JWT authz sweep, like the
collector blueprint. 8 reporting tests; full suite green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First execution phases of docs/proposals/ge-enforce-plugin.md. The GE-Enforce
manifest becomes shopdb data.
P0 scaffold: new geenforce plugin (api_prefix /api/geenforce, default_enabled
false, core_version >=0.7.0). Registered in PLUGIN_TABLE_OWNERS (ADR-008); its
0001 baseline really creates the tables.
P1a model: one wide manifestentries table + entrytype discriminator (not STI,
not JSON blob), manifestscopes (UNIQUE scopename+phase), the three multi-value
filter child tables, inusechecks + processes, immutable manifestpublishedversions
(frozen rendered JSON), manifestpayloads (inline, capped), pctypealiases
(mirror of the engine lib's alias graph). regvalue stored as its raw JSON
literal so DWord typing survives.
P1c importer + exporter: parse common + gea-shopfloor-* + preinstall.json into
draft rows and rebuild the JSON verbatim from rows in sortorder.
P1d parity harness (GATE A): filters.py mirrors the engine's four filter
functions + alias graph; parity.py proves import+export is behaviorally lossless
(field-identical + same-entries-fire across 18 machine-profile fixtures) WITHOUT
byte-diffing. Verified PASS against all 11 real reference manifests (64 entries)
and a synthetic site-neutral fixture covering every type/filter (the CI gate).
First slice (gea-shopfloor-cmm shape): service layer (import/publish/rollback/
export-to-share), CLI (parity, import-share, publish, export-share), and the
client endpoint GET /api/geenforce/manifest serving the current published
snapshot (never the draft) with ETag/304. Split permissions
geenforce.manage/publish/fetch. Tests prove import->publish->serve, draft edits
never change served bytes, publish+rollback, and auth (401 unauth/wrong-scope).
Contract 0.11.0: added service_token_authorized(scope) to shopdb.api so plugin
service endpoints authorize a scoped managed token without importing core token
internals. Documented in PLUGIN-HOOKS.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Every publish is a permanent immutable revision (manifestpublishedversions),
kept indefinitely; optional retention policy (keep last M / prune older than N)
deferred, default keep-everything.
- Draft edits are not versioned (working copy overwrites), so field-level "who
changed what between publishes" rides the existing core audit system - no new
table, shows in the Audit Logs UI IT already uses.
- Runbook: History tab for published versions + roll back; Audit Logs for draft
edit provenance.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an execution plan and simplify the design for average-site-IT operability
(the governing constraint from the review):
Model simplifications:
- One wide manifestentries table with an entrytype discriminator, not SQLAlchemy
STI subclasses and not a JSON blob. ~64 total entries fleet-wide make sparse
columns free and keep rows readable in plain SQL.
- Published snapshots freeze the rendered JSON document in a single manifestjson
column; drop the row-mirrored manifestpublishedentries family. Immutability is
structural, rollback is a one-flag flip, diff is a text diff.
- New manifestpayloads table for inline bytes with a ~1 MB app cap.
- regvalue stores the raw JSON literal (DWord typing); applymode/updatewindow
flagged inert-in-engine so the UI labels them.
Execution plan (section 13):
- Phases P0-P6 with gates; parity harness spec (two checks, IT-readable output,
~16-18 machine-profile fixtures); first vertical slice through
gea-shopfloor-cmm; ranked fail-fast risks.
- Milestone 1 = author + publish in shopdb, export to the share by a button,
engine/dispatcher/PCs unchanged. Real pain relief at zero client risk, with a
rollback IT already knows (restore the _meta/history backup).
- Export-to-share promoted to a first-class feature and permanent break-glass.
- Split permission geenforce.manage (edit) vs geenforce.publish (ship).
- Move Up/Down instead of drag-and-drop; a "what would this PC get" simulator
endpoint + UI; three-increment editor build.
- Two-source pctypemap transition window; scope-inventory reconciliation
(gea-shopfloor-display has no share dir).
- Plain-English IT day-to-day runbook proving the design is manageable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fold six review findings into docs/proposals/ge-enforce-plugin.md:
- Parity gate is behavioral equivalence, not byte-identity. Re-serialized JSON
differs in key order/whitespace/_comment formatting, so a raw diff never
converges; the test is same ordered entry set with identical detection/
targeting/action per entry.
- Dedicated payloadsha256 column, independent of detectionmethod. DetectionValue
is a SHA256 only for detectionmethod=Hash; MSIs with Registry/FileVersion
detection carry no payload hash, so an HTTP/inline fetch would otherwise run
unverified bytes. Client verifies fetched bytes against payloadsha256.
- Immutable published snapshots (manifestpublishedversions). Editing touches a
draft only; publish freezes a snapshot; the client is always served the latest
published snapshot, never the live draft; rollback republishes a prior
snapshot (the post-cutover safety net once the on-share JSON is retired).
- Scope uniqueness is (scopename, phase), not scopename alone; preinstall is one
flat scope gated internally by PCTypes, not per-pctype scopes.
- Alias graph: engine lib stays the single source of truth, shopdb only mirrors
it for validation; do not invert to engine-fetches-from-shopdb.
- Desired-vs-observed needs a new collector field (the installedVersions status
map), not existing data; flagged as a dependency.
Plus TLS trust for the SYSTEM-context client and importer skips .bak variants.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Metrology PCs (CMM, Keyence, Genspect, wax-and-trace imaging pc-types) drive
an attached measuring instrument. The PC itself stays a shopfloor PC, but the
collector now models the instrument:
- New METROLOGY_TOOL_MAP (pctypemap.py) maps those pc-types to a
MeasuringToolType (CMM, Vision System, Genspect, Form Tracer).
- ComputersPlugin._sync_measuringtool_link creates the MeasuringTool asset
once and a directional PC->tool "controls" relationship, tagged
collector:measuringtool. Idempotent (re-push reuses, no duplicate asset) and
self-archiving (a PC re-imaged to a non-metrology type deactivates the link
but keeps the asset and any calibration history). Mirrors the printer-link
pattern. The MeasuringToolType is created on demand if not seeded.
- 4 tests: create+link, idempotent re-push, non-metrology skip, repurpose
archives. Non-metrology PCs never warn about a missing controls type.
Settings rail cleanup:
- Collapsible groups so the 13-group rail fits without scrolling (1511px ->
488px). The group containing the current page expands; the rest collapse.
CSS-drawn caret (ASCII source, no Unicode). Empty groups never render, in
both the rail and the landing page.
- Measuring Tools group placed with the other asset groups (right after
Machines) instead of appended last; empty placeholder positions the
plugin-contributed cards.
- Operating Systems moved from PCs to General Reference: OS is cross-asset
(PCs, machines, measuring tools, network devices all run one).
Plus docs/proposals/ge-enforce-plugin.md: a planning doc for refactoring
GE-Enforce/DSC into a shopdb plugin (manifest as shopdb data, payloads on
SMB/HTTP/inline), grounded in the real manifest schema.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
List pages kept the current page in local state, so clicking into an
asset and hitting Back remounted the list at page 1. A shared
useListQuery composable now mirrors the page (and search term) into the
URL query via router.replace across all 18 list pages, so Back restores
the page you were on and lists are deep-linkable. Page 1 with no search
stays a bare path; changing a filter resets to page 1; unrelated query
keys are preserved.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
One canonical card order applied to machines, PCs, printers, network,
and measuring tools: Identity -> type-specific -> status -> Location &
Organization -> domain -> Custom Fields -> Warranty -> Relationships ->
Notes -> audit footer, with a documenting comment on each page so they
stop drifting. The location card is Location & Organization everywhere;
the network detail page is rebuilt into the family (Asset Information
folded into Identity, Record Info retired for the standard audit
footer, a Location & Organization card added). Fixed two latent bugs
found in the process: printer detail had no audit footer, and the
network Record Info read datecreated/datemodified (not in the payload)
so its timestamps rendered blank - the footer now uses the correct
fields.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collector: the computers collector schema gains defaultprinter and
printers; apply_collector_payload resolves each reported identifier to
a printer asset (windowsname/hostname/sharename/assetnumber/IP,
first-hit case-insensitive) and idempotently syncs relationships -
defaultprinter (directional) for the default, connectedto for the
rest. Collector-created rows are tagged so a re-report archives dropped
links while manual relationships are never touched; unresolved
identifiers warn instead of failing. Both PC and printer detail pages
show the links via the shared relationships card (no frontend change).
GE-Enforce Win32_Printer collection snippet documented.
Searchable custom fields: a per-field searchable flag (migration 7d24);
global search matches custom-field values on flagged active fields and
routes each hit to the asset detail page, reusing the existing
(type,id) dedupe and search_<type>_enabled domain filter. Searchable
toggle on the Custom Fields settings page.
822 tests pass; both verified live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The content grid hand-assigned cards to a fixed left/right split, so a
PC or machine with many tall cards (installed apps, warranty,
relationships) piled them all on one side. Switched .content-grid to a
balanced CSS multicolumn flow (display:contents flattens the wrappers
so no markup changes), with break-inside:avoid keeping cards intact.
Cards now distribute by height and the columns stay even on every asset
detail page.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The PC detail Installed Applications section 500d and vanished on any
real PC: ComputerInstalledApp had no to_dict, so the endpoint errored
and the v-if hid the section. Added the serializer (curated version
wins over the raw collected string, app name + description included)
and aligned PCDetail to the flat payload; regression test added.
Also: employee detail skips its USB panels when the usb plugin is
disabled (was firing 404s), and the shopfloor kiosk header is now
light-on-dark for readability.
810 tests pass; PC 259 installed apps verified live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The single-label page gains an Output toggle (standalone vs place on a
ULINE 6-up sheet) with a 2x3 cell picker, so one label can be printed
into the correct physical position on a partially-used sheet - the
single-label equivalent of the batch page start-cell offset. Batch page
unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Cuts the large post-0.6.0 pile as a pinnable release: the machines
rename (ADR-011), the API import surface, personal/scoped/collector
API tokens and the get_permissions plugin hook, the ADR-010 frontend
hook contract, per-plugin migrations, model/employee photos, the
dualpath single-machine toggle and relationship propagation, support
teams, email sending, and the shared asset label generator. Plugin
contract moved 0.6.0 -> 0.10.0 over this range (distinct series per
ADR-007).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adding the Change Password action turned the user menu into a
full-width-button row that overflowed the fixed sidebar. The footer is
now an identity row (avatar + truncating name) above a compact
icon-button actions row (Password / Logout), both within bounds.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A multi-select batch sheet at /print/asset-label-batch/<type> lays
selected assets onto ULINE label pages (6-up 3x3 or dense 72-up mini),
with code-type and encode toggles matching the single label, plus a
start-cell offset for partial sheets. Print Labels buttons on all five
asset list pages. The per-type config and encode resolution are
extracted to a shared print/assetLabel.js used by both the single and
batch views. Measuring-tool batches encode each tool inspection-
operation code (decode-verified 0615).
808 tests pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Email: a stdlib SMTP mailer (settings-first config, graceful no-op when
unconfigured), a test-email endpoint wired to the Email settings page,
forced first-login password change (users.mustchangepassword, migration
7d23, /change-password flow), new-user welcome mail, and on-demand
report/alert delivery (POST /api/reports/email + Email Report buttons)
with an external-cron-with-a-scoped-PAT path documented for automation.
All tests patch smtplib - no network.
Labels: a shared /print/asset-label/<type>/<id> view any asset detail
page opens - card or plain style, QR or barcode, configurable encoding.
Per-type qr_target_* templates plus label_default_style/codetype/encodes
settings on the Printing page. Measuring-tool labels default to encoding
their inspection-operation code (derived from the location name, e.g.
0615), so every tool in an area shares the area code - verified by
decoding the rendered QR. Machine labels default to the machine number;
blank-serial handled gracefully.
808 tests pass; both features verified live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Support teams: contact management moved from a row expander to a modal
(Contacts (N) button per team); application detail Support card and the
modal show Email (mailto) and Teams chat buttons for contacts with an
SSO, derived as sso@ + a new contact_email_domain site setting
(default geaerospace.com, blank hides the buttons).
Audit log: hovering a user SSO shows the full name, resolved
best-effort from the employee directory in either mode.
Docs/hygiene from a standards review: CLAUDE.md active-state,
CONTRACT-STABILITY.md and README brought to contract 0.10.0 / 11
plugins / migration head 7d22; get_asset_panels endpoint path fixed in
the hook docstring; leftover debug console.logs removed.
781 tests pass; contacts modal, action-button hrefs, and the audit
tooltip verified live.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Audit Logs now uses a fixed table layout with tuned column widths:
every column fits the card without horizontal scrolling, long entity
names truncate with a hover tooltip, timestamps show a compact
no-seconds form (full value on hover), and the 9-digit user SSO renders
fully.
Removed the Collector service token quick-preset from the create modal
per user preference - the collector.ingest checkbox in the scope grid
is the path now.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The 401 interceptor (session expired) redirected to bare /login,
losing the page the user was on - the router guard already preserved
it but the expiry path bypassed the guard. The interceptor now carries
?redirect= like the guard does, and Login returns there.
The API tokens tables overflowed the card: the token column no longer
repeats the shopdb_pat_ prefix per row (short prefix shown, full form
on hover) - the token itself was already never displayed after
creation.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plugins declare their own RBAC permissions instead of core accumulating
them: 36 permissions moved out of the core catalog into the 9 owning
plugins (core keeps the 19 its own blueprints enforce). The catalog is
resolved dynamically (core + enabled plugins) and feeds the roles grid,
the token scope picker and ceiling, and flask seed permissions;
installing or enabling a plugin seeds its permissions automatically. A
disabled plugin drops out of the assignable catalog while existing role
links keep working. New plugins - bundled or external - now bring their
permissions with zero core edits.
781 tests pass; live-verified with a machines.edit-scoped token.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A token scoped to the new collector.ingest permission is a collector
service token: the collector endpoints accept it via X-API-Key or
Bearer alongside the env fleet keys (which remain the fallback), giving
the fleet credential rotation, revocation, and last-used visibility
from the API Tokens page. Containment holds both ways: a collector
token authorizes nothing else, and no other credential gains collector
access. Shared token validation refactored out of the auth shim; a
Collector service token quick-preset in the create modal; integration
guide documents minting, rotation via site-config.json, and the
service-identity pattern.
765 tests pass; live acceptance matrix verified.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A token may carry a scopes list: it then grants only those permissions,
intersected with what the owner holds at use time, with the admin role
bypass suspended and role-gated routes denied - a scoped token from an
admin account is genuinely limited. Scope ceiling enforced at
create/update too (only permissions the owner holds; 400 lists
violations) and the picker only offers what you hold. Token management
itself now requires the new apitokens.create permission (admin by
default, grantable via roles). Unscoped tokens keep the exact prior
act-as-owner behavior; imports need an unscoped admin token.
Migration 7d22.
756 tests pass; live-verified scoped 201/403 matrix.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
API tokens: any user mints named, optionally-expiring tokens
(shopdb_pat_..., sha256-stored, secret shown once) at Settings > API
Tokens; a before-request shim swaps a valid PAT for a request-scoped
JWT of its owner, so the entire existing auth/authz/import-mode stack
works unchanged and revoked/expired tokens 401 cleanly. Built for
long-running scripts - the legacy import no longer dies when a login
JWT expires. Migration 7d21_apitokens; create/revoke audit-logged.
Audited integration gaps fixed: Asset.to_dict serializes measuring
tools (typedata + pluginid - relationship links to tools resolve); map
subtype filter/colors and MapEditor include them; dashboard totals
count them; warranty links use a new by-asset route; the measuringtools
ADR-010 hooks are real (corrected presentation token, implemented
map-overlay endpoint); the login avatar resolves through the
employee-photo helper.
737 tests pass; naming green; frontend builds; both features verified
live end-to-end.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The measuringtools plugin was missing from two cross-cutting surfaces:
the asset-identifier matrix (no measuring_tool column or per-type
keys - gauge lab reference is their primary identifier) and global
search (results fell to a generic URL and gaugelabreference was never
searched). Measuring tools now have identifier toggles, gated
gauge-lab and maintenance-reference fields on their form and detail,
a search domain toggle, gage-tag search, and proper labels, routes,
and filter chips in search results.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Keying the outer router-view on route.path made every settings click
remount the whole settings shell, resetting the rail scroll to the
top. The key now treats /settings/* as one unit so the shell (and its
scroll position) persists while child pages swap; detail-to-detail
remounts elsewhere are unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fields rendered in arbitrary key order (dualpath toggle first,
facility name buried, setup flag mid-list). Now grouped with headers:
Identity, Behavior, Naming and Patterns, Data Sources, System - with
unknown future keys falling into Other at the end.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The monolithic tab page competed with the settings rail as a second
navigation system, and its Integrations tab was a dumping ground. Each
section is now its own routed rail page (ServiceNow, Zabbix Supplies,
Dell Warranty, Collector PC Types, Branding, Floor Map, Printing and
Labels, Email/SMTP, Audit, Authentication, Asset Identifiers, Global
Search), thin over a shared useSystemSettings composable, grouped
logically in the rail with system groups clustered last. Old
/settings/system?tab= URLs redirect to the right page.
Also fixes the post-login redirect: the auth guard now remembers the
intended destination and Login returns there (same-site paths only).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Booleans render as checkbox toggles, the employee/USB directory-mode
settings as selfhosted/external dropdowns, and every field has a label
and help text (raw keys and type-true/false text boxes are gone).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Computer and Printer payloads now surface the linked model imageurl the
way machines already did, and the machine/PC/printer detail heroes
render the photo when present (network devices and measuring tools
have no model link, so nothing to surface). Absent images render
nothing rather than a broken icon.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Most facilities consider a Dualpath pair one physical dual-bay machine.
New site setting dualpath_single_machine (default on): the machines
list, dashboard counts, machines-by-type report, and floor map collapse
each pair to its primary bay (lower assetnumber), with combined
2007 / 2008 labels; pagination totals stay honest. Detail pages remain
per-bay and always show a dual-bay sibling banner linking the partner.
Pair resolution lives in core services and joins the plugin contract
surface (0.8.0 -> 0.9.0).
On the WJ dataset: 31 pairs collapse, machine counts 262 -> 231, map
470 assets. Toggle verified live in both states, left on.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
activeTab read ?tab= only once at mount, so settings-rail links that
change just the query (Branding, Floor Map) updated the URL without
switching the panel. The query param is now the source of truth: a
watcher applies rail/back/forward navigation and tab clicks write the
query via router.replace.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Symmetric relationship types (isdirectional flag, migration 7d19) show
one entry per peer on the relationships card - a Dualpath pair no
longer lists its partner twice - and directional types read naturally
instead of Outgoing/Incoming. Deleting a collapsed entry removes every
underlying direction row.
Propagation is now real (migration 7d20): relationship types declare
propagation-through pairs in relationshiptypepropagations (M:N,
replacing the never-consumed single column); creating a controls link
on either bay of a Dualpath pair auto-creates it on the partner,
mirrored across both endpoints because live data stores controls as
bay -> PC. flask relationships propagate backfills existing data (29
rows fanned out on the WJ dataset, idempotent).
This also completes the tree that commit 1d21bf0 accidentally split
(core/models/__init__ imported RelationshipTypePropagation ahead of the
file that defines it), returning CI to green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Model photos: upload/replace/delete on /api/models/<id>/image (admin),
stored under instance/modelimages/ with a public serve route; thumbnail
plus Upload/Replace/Remove controls in the Models settings modal; the
URL field remains as a manual alternative.
Employee photos, mode-aware: self-hosted directory employees get
upload/replace/delete (photo-<sso> under instance/employeephotos/,
employees plugin migration 0002); external directory mode passes the
HR-supplied picture URL through read-only (writes 409). One resolver
feeds both consumers - the shopfloor recognition/recert kiosk cards and
the employee detail hero - in either mode.
Navigation fix: router-view is keyed on route path, so following a
relationship link between two assets of the same type (machine ->
dualpath machine) reloads the page instead of showing stale content;
query-only URL changes still avoid a remount.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the legacy supportteams/appowners pair: supportteams
(teamname unique, teamurl ServiceNow link) + supportteamcontacts
(multiple named contacts with SSO per team, the people you reach out
to), applications.supportteamid intact. Migration 7d18 migrates each
legacy team owner into a contact, drops appowners, and has a validated
downgrade. New /api/supportteams CRUD (admin writes, import-mode
timestamps, teamname lookup), Support card on application detail,
contacts column on the list, and a settings management page.
IMPORT-API.md mapping updated to the concrete endpoints.
658 tests pass; live dev migration applied (24 teams / 24 contacts);
fresh-install and downgrade round-trips verified on scratch DBs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Goal: an LLM or script can migrate an entire legacy database using only
the HTTP API - original history preserved, safely re-runnable.
- X-Import-Mode header (admin only): create/update endpoints across 15
timestamped entity types accept original createddate/modifieddate;
helper exposed via shopdb.api (contract 0.7.0 -> 0.8.0).
- Exact-match natural-key lookup filters on 13 list endpoints for the
lookup-then-upsert recipe.
- Selfhosted USB checkout/checkin accept backdated event times in
import mode.
- docs/IMPORT-API.md: operator manual grounded in the real legacy
schema - order of operations, full table-by-table mapping including
the machines fan-out, idempotent Python importer with dry-run, parity
checks, and decided dispositions for unmigrated tables (DNC config
stays live-fed via the collector; supportteams/appowners map to the
upcoming supportteams model).
635 tests pass; naming green; frontend untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The sweep exercises usb/employees handlers, which default to EXTERNAL
directory mode when no setting row exists - green on the dev box where
the external MySQL databases happen to exist, red in CI where they do
not. Seed selfhosted mode in an autouse fixture so the guard is
deterministic everywhere. Verified by running the suite with the
external DB hosts pointed at an unreachable address.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Four data-only hooks on BasePlugin (get_settings_cards,
get_asset_panels, get_map_overlays, get_asset_presentation) with a
GET-only /api/pluginui consumer surface copying the dashboard-widgets
semantics. Pilots: warranty declares its asset panel; measuringtools
supplies its settings card, presentation, and calibration overlay -
the last hardcoded settings-nav entry is now hook-sourced. Generic
renderers for panels/overlays/presentation deferred per the ADR's
incremental adoption plan (documented in CONTRACT-STABILITY.md).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every non-exempt POST/PUT/PATCH/DELETE must 403 a role-less member
and pass authz for admin; exemptions (auth, collector, setup wizard,
kiosk click-through, admin-or-self user update) are documented in
the test. Any future unguarded mutation fails CI as its own case.
Sweep confirmed existing gating complete: zero routes needed fixes;
lockout already implemented.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Bump __version__ per ADR-007; roll Unreleased into the 0.6.0
changelog section. Covers the machines rename (ADR-011), the
measuringtools plugin, per-plugin migrations (ADR-008), route
gating (ADR-009), the get_reports hook (contract 0.6.0), and
the sister-site adoption docs.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The curated Controlled Machines (PC detail) and Connected PC (machine
detail) cards duplicated what the generic relationships card already
shows with type badges and inline add/remove - and hid every
non-controls relationship type. Removed the curated cards and their
plumbing (~105 lines of orphaned CSS included); the MachineForm
controlling-PC picker stays as the write path. Also theme-variable
fixes on the shared card styling.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The equipment plugin is now the machines plugin, ending the UI-vs-code
vocabulary split while the contract is pre-1.0 and nothing external
depends on the old names.
- plugins/equipment -> plugins/machines: manifest, class, /api/machines,
machines.* permissions, registry key (with an auto-migrating load shim
for existing installs).
- Tables: equipment -> machines (equipmentid -> machineid) and
equipmenttypes -> machinetypes, renamed in the plugin's own migration
chain (machines0002rename), idempotent for both upgrading and fresh
installs.
- The legacy core machinetypes lookup actually types the vendor MODELS
catalog, so it is renamed losslessly to modeltypes
(models.modeltypeid, /api/modeltypes, Model Types settings page)
rather than collapsed, freeing the machinetypes name. Core migration
7d17_machines_rename also flips data in place: assettypes row
equipment -> machine, auditlog entitytype, identifier_/search_
settings keys, permission rows, and renames alembic_version_equipment.
- Frontend: machinesApi/modeltypesApi, item.machine response shape,
assettype value compares 'equipment' -> 'machine' (map, search,
custom fields, relationships), routes machines.js with plugin gating
retagged, /print/machine-badge, Machine Types (subtypes) and Model
Types (catalog) settings pages, machines-by-type report id.
- Docs swept; ADRs left as history per the authoring rule.
Upgrade: flask db upgrade then flask plugin upgrade-all.
Verified: dev DB flipped live (262 machines, 35 modeltypes, 95 models
retyped, zero equipment tables remain); fresh scratch-MySQL install
produces the new names; 341 tests green; naming/style green; frontend
builds; live E2E on machines list/detail, PC relationships, map,
reports, and both settings pages.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Records the approved domain rename (equipment plugin/tables/API ->
machines) and the lossless retyping of the models catalog (legacy
machinetypes -> modeltypes), with migration placement per ADR-008.
Implementation lands in the following commits.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Relationship type names are data and the seeded rows mix casings
(controls, Controlled By, connectedto); the PC and machine detail/form
pages compared === Controls and silently showed no relationships.
Compares are now case-folded in all 7 sites.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Display label only, matching the Machines relabel; the endpoint id and
equipmenttype entity are unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ADR-010 (PROPOSED): four data-only frontend hooks following the
get_reports precedent - get_settings_cards, get_asset_panels,
get_map_overlays, get_asset_presentation - batched as a future contract
0.7.0. Warranty proves asset panels first, measuringtools the rest;
accept only after both run on the hooks.
Display-label swap Equipment -> Machines across nav, list, form,
detail, search, map editor, and settings copy (incl the API-failure
fallback nav). Identifiers unchanged: plugin name, tables,
/api/equipment, asset type, and routes all stay.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Path.rglob does not descend symlinks, so a symlinked external plugin
(the ADR-003 dev loop) silently escaped the contract-purity scan. The
scanner now resolves plugin dirs before walking, a regression test
plants a symlinked plugin with a real violation and asserts it is
flagged, and the known-limitation notes in the external-repo docs are
lifted.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- flask plugin new now scaffolds the frontend too: List/Detail/Form
views on the global styles, a gated route module (ADR-009), and an
api-client snippet emitted into the plugin dir. Views are written
before the route file so a partially generated plugin cannot 500 the
dev server.
- docs/PLUGIN-EXTERNAL-REPO.md + scripts/test-external-plugin.sh: how a
sister site develops a plugin in its own repo and runs the framework
contract tests in CI against a pinned framework ref (script verified
to fail on a broken core_version pin).
- docs/CONTRACT-STABILITY.md: settled vs churning contract surface and
the provisional 1.0 criteria.
- CLAUDE.md active-state refresh (contract 0.6.0, 11 plugins, 340
tests, measuringtools done).
Known limitation documented: Path.rglob does not descend symlinks, so
the import-surface contract test skips symlinked external plugins.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PLUGIN-QUICKSTART Step 7 is now a concrete 7-item checklist (view
conventions, route auto-discovery + ADR-009 gating meta, api client
shape, nav/report hooks, settings auto-nesting, verification).
New tests/test_docs_contract.py introspects BasePlugin and fails CI when
a public hook is missing from PLUGIN-HOOKS.md or the documented contract
version drifts - it immediately caught two undocumented hooks
(get_provisioning_note, get_config_schema), now documented.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The test suite silently depended on the gitignored dev
instance/plugins.json to know which plugins to load - green on the dev
box, red on any fresh clone (CI caught it). conftest now seeds a
registry enabling all bundled plugins when none exists; an existing dev
registry is left untouched.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Gitea runner runs jobs on the host, where actions/setup-python has
no prebuilt 3.12 for the distro. Use the system python3 in a venv.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Gage-lab instruments as Asset extensions: measuringtooltypes (color-coded
lookup) + measuringtools (calibration interval/dates, provider, notes) with
calibration status derived at read time (overdue / due soon / current /
unknown), never stored. Full CRUD API with permission-gated writes, types
management with in-use guard, calibration report, nav/reports/config-schema
hooks, and a complete frontend (list/detail/form, types settings page,
calibration report page, gated routes per ADR-009).
First plugin whose migration chain really creates tables post-cutover
(ADR-008), and the working example for docs/PLUGIN-GUIDE.md - a 12-section
walkthrough of building a plugin on this framework, linked from
PLUGIN-QUICKSTART and PLUGINS.
Verified: full suite 323 passing, live E2E on all four pages, fresh
scratch-MySQL migration dry-run green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Framework:
- Per-plugin Alembic migration chains (ADR-008): every bundled plugin
carries its own chain with a stamp-only anchor at the ownership cutover;
new plugin schema lands in plugins/<name>/migrations/, never the core
chain. Deploys add flask plugin upgrade-all. Fixed a latent bug in the
shared alembic template (engine URL resolution) and taught the metadata
filter to include FK-referenced core tables.
- Frontend plugin route gating (ADR-009): plugin routes carry meta.plugin;
a disabled plugin's pages redirect to the dashboard via a cached,
fail-open check against the new public GET /api/plugins/enabled.
- get_reports() plugin hook (contract 0.5.0 -> 0.6.0): plugins contribute
report cards; warranty and toner cards moved off the hardcoded list.
Reports:
- Hub grouped by category with search; inline reports render at the top,
are URL-backed (?report=id, back-button and deep links work), expose
their server-side filter params as controls, and export CSV. Warranty
and Toner pages gained CSV export.
- Deleted the dead legacy Warranty Status report (always-zero buckets
from a retired column).
Theming and fonts:
- Inter (variable) bundled locally via @fontsource, replacing the Google
Fonts Roboto import - air-gapped installs now render correctly; tables
use tabular numerals.
- Optional brand_primary_dark_color, brand_accent_color,
brand_sidebar_color settings applied to CSS vars at bootstrap.
USB frontend repair (views were reading a dead legacy shape):
- List/detail/form and the employee profile USB panels remapped to the
real API shape (device_id/device_desc/checkinoutlog); employee panels
now use /usb/checkouts endpoints; external-mode /usb/checkouts/active
honors the badge filter; dead client methods pruned.
Also: warranties list page no longer requires login (matches app
convention); collector doc rewritten with a GE-Enforce integration guide
and paste-ready PowerShell reporter; ADR index and CHANGELOG updated.
Verified: 323 tests pass, naming/style green, frontend builds, plugin
migration dry-run green on scratch MySQL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-11 10:01:47 -04:00
1110 changed files with 223737 additions and 18605 deletions
Follow these when suggesting code. They are enforced by a naming/style hook and
by CI (`.github/workflows/ci.yml`) - suggestions that break them fail the build.
`CONTRIBUTING.md` is the full authority; this is the short version.
## General Guidelines
- Finish the in-flight task before starting new workstreams.
- When asked to "take notes" on a future idea, capture it in the notes file only - do not implement it, and do not expand on it at length in replies.
- Use a temp file for multi-line git commit messages instead of heredoc syntax: `git commit -F <file>`.
- Avoid using `powershell.exe -File` for launching scripts, as it fails if the script file is missing or quarantined.
## Installer Logic
- The endpoint agent installer shipped to endpoints is generated by `_install_script()` in `plugins/endpoints/api/siteagent.py`, NOT `agent/installer/install-shopdbagent.ps1`. Any installer logic change must be made in `siteagent.py`, followed by rolling the containers.
- The user's workflow: the agent hands over only the built EXEs (`shopdbagent.exe`, `shopdb-endpoint-tray.exe`); the user uploads them via the "Upload New/Legacy Agents" field in the Build Site Agent UI and builds the package with the newest version selected at the top.
## Naming (LOCKED - the hook rejects violations)
- **DB tables**: lowercase, concatenated, plural. No underscores, no dashes.
`machines`, `networkdevices`, `businessunits` - NOT `machine_types`, `BusinessUnits`.
- **DB columns**: lowercase, concatenated, singular. No underscores.
`machineid`, `lastzabbixsync`, `isactive` - NOT `machine_id`, `last_zabbix_sync`.
with quantities are standalone tables, not assets.
- **Ledger pattern**: a cached `quantityonhand` moves in the SAME commit as the
signed transaction row it reflects.
- **Configuration for endpoint agent**: The endpoint agent's live collector key and base URL are provisioned in the Windows registry `HKLM\SOFTWARE\GE\ShopDB` (values `CollectorKey`, `BaseUrl`), not committed in `appsettings.json`; `appsettings.json` holds only non-secret runtime settings.
## Before you finish a change
Run the three gates (CI runs the same):
```
python -m pytest tests/ -q
cd frontend && npx vitest run && npm run build && cd ..
bash scripts/check-naming-and-style.sh
```
Commits: short present-tense subject, body says WHY. No AI/tool attribution in
- Single core Alembic chain: baseline `68b3947ae14f` -> head `7d16_directoryemployees` (23 migrations). A fresh site runs `flask db upgrade` from empty; it is reproducible and idempotent.
- Pre-1.0 framework; sister sites should pin tight `core_version` ranges until contract reaches 1.0
### Deferred
- Equipment data migration (one-shot script for legacy ASP shopdb -> assets). Per ADR-001, only `category='Equipment' AND machinenumber IS NOT NULL` migrates. Skill `migrating-asset-schema` documents the pattern; the actual one-shot script lives in `scripts/migration/` when run.
- Printers retirement: legacy `PrinterData` model + frontend changes. Coordinated with the equipment data migration.
-`measuringtools` plugin (ADR-005). First plugin to be built using the scaffold.
- Frontend hook contract for asset-detail, map markers, search results
- Alembic per-plugin migration chains (the framework supports them; bundled plugins haven't moved off `db.create_all()` yet)
## Quick start
```bash
# Start dev environment
~/start-dev-env.sh
# Activate venv and install deps
cd /home/camp/projects/shopdb-flask
source venv/bin/activate
pip install -r requirements.txt
# Configure environment
cp .env.example .env
# Edit .env with DB credentials, JWT secrets
# Create / update database tables via the Alembic chain
flask db upgrade
# Seed RBAC, default settings, and reference data (all idempotent)
flask seed permissions
flask seed settings
flask seed reference-data
# Restart services
pm2 restart shopdb-flask-api shopdb-flask-ui
```
## Service URLs
- Flask API: http://localhost:5001
- Flask UI: http://localhost:5173
- Legacy ASP (data source for one-time import): http://192.168.122.151:8080
`CONTRIBUTING.md`is authoritative: lowercase concatenated DB columns, snake_case
pure-code identifiers, a closed acronym list, banned shorthand. The pre-commit
hook `scripts/check-naming-and-style.sh` enforces what it can.
A modern rewrite of the classic ASP/VBScript ShopDB application using Flask (Python) and Vue 3. This application manages shop floor machines, PCs, printers, applications, and related infrastructure for manufacturing environments.
## Overview
ShopDB tracks and manages:
- **Machines** - CNC equipment, CMMs, inspection systems, etc.
-`type` - Filter by machine type (pc, printer, equipment)
## Plugin System
ShopDB supports plugins for extending functionality. See `CONTRIBUTING.md` for plugin development guidelines.
The image bundles ten plugins; only the ones a site installs are loaded:
- **computers** - Shopfloor PCs and workstations
- **employees** - Employee directory
- **equipment** - CNC, CMM, and inspection equipment
- **knowledgebase** - Documentation and troubleshooting guides
- **network** - Network devices
- **notifications** - Shopfloor notifications and recognition feed
- **printers** - Extended printer management with Zabbix integration
- **slides** - TV/kiosk slideshows
- **usb** - CMMC USB check-in/out tracking
- **warranty** - Dell warranty lookups
## Legacy Migration
This project replicates functionality from the classic ASP/VBScript ShopDB site. Key mappings:
| Legacy | Modern |
|--------|--------|
| ASP/VBScript | Flask/Python |
| Classic ADO | SQLAlchemy |
| Server-side HTML | Vue 3 SPA |
| Session auth | JWT tokens |
## License
Internal use only.
# ShopDB Flask
A modern rewrite of the classic ASP/VBScript ShopDB application using Flask (Python) and Vue 3. This application manages shop floor machines, PCs, printers, applications, and related infrastructure for manufacturing environments.
## Overview
ShopDB tracks and manages:
- **Machines** - CNC equipment, CMMs, inspection systems, etc.
flask run --port 5001# MUST be 5001 - the frontend dev server proxies here
# Frontend (separate terminal)
cd frontend
npm install
npm run dev # dev server on :5173
npm run build # production build into frontend/dist (served by Flask)
```
Complete first-run setup at `/setup`, or run `flask seed admin` for a headless
admin account. The repo ships VS Code config in `.vscode/` (F5 debugs the
backend; a "Dev site" task runs both servers). A fuller day-one walkthrough,
including VS Code and troubleshooting, is the DEVELOPMENT-SETUP page in the
project wiki.
To import a site's legacy data, use the HTTP import surface: an admin API
token plus the IMPORT-API page in the project wiki drive the whole migration
through documented endpoints (`X-Import-Mode` preserves original timestamps).
`scripts/site_imports/wjf/` is the reference site's loader, kept as a worked
example of the whole sequence.
### Which deployment route
| Target | Use |
|---|---|
| **Windows Server + IIS** (how sister sites run) | **the INSTALL-WINDOWS page in the project wiki** - one installer `.exe`, offline, no manual IIS work. Day 2: the OPERATE-WINDOWS page in the project wiki |
| Linux / Docker, air-gapped | the DEPLOY-AIRGAP page in the project wiki |
| Linux / Docker, connected | the DEPLOY page in the project wiki |
For every environment variable and Setting key see the CONFIG page in the project wiki.
### What the Windows installer requires
The installer checks all of this before it changes anything and refuses rather
than half-installing.
| | Minimum | Notes |
|---|---|---|
| Windows Server | **2019** (build 10.0.17763) | Ships **IIS 10.0**, which is therefore the earliest IIS supported. Server 2016 is also IIS 10.0 but falls below the build floor and is refused. |
| Windows client (test boxes) | **10 22H2** (build 10.0.19045) or 11 | **Pro or higher** - Home has no IIS at all. |
| Architecture | 64-bit | The payload is cp314 win_amd64. |
| IIS Web Server role | installed beforehand | `Install-WindowsFeature -Name Web-Server -IncludeManagementTools`. Needs no internet. |
| Free disk | 5 GB | Refused below this. 40 GB is comfortable once uploads and config backups accumulate. |
| Database | bundled MySQL 8.4 LTS, or your own | Bring credentials if you use an existing server. |
The binding constraint is the operating system rather than IIS: the payload is
Python 3.14 plus HttpPlatformHandler and URL Rewrite, and the handler itself runs
on older IIS. An older host is untested rather than known-broken.
If you must deploy onto something older, the INSTALL-WINDOWS-IIS page in the project wiki
is the manual procedure and has no OS gate - it even documents MySQL 5.6, which is
that era of machine. The trade is stated there: it produces a server the
"_comment":"The profile the released Windows installer is built from. Every bundled plugin that carries a manifest, so ONE exe serves any site: the wizard offers all of them and the operator ticks what that site uses. Plugins left unticked are never installed, and 'flask plugin prune-schema' drops their tables at provisioning (ADR-014). 'applications' is deliberately absent - it is manifest-less core and always ships. Build with: deploy/windows/installer/build-installer.sh deploy/site-profile-universal.json <repo>. Use site-profile.example.json instead only when a site genuinely needs a lean build; see ADR-013."
"_comment":"Declarative plugin selection for a site (ADR-013). Apply with: flask plugin apply-profile deploy/site-profile.example.json. 'plugins' is the set this site wants; their hard dependencies are pulled in automatically and everything installs + enables in dependency order (idempotent). Naming a plugin not on disk fails loudly. 'locked' is reserved for a future guard against removing a site-mandated plugin. apply-profile never removes plugins absent from the list - removal stays an explicit flask plugin uninstall. Run flask plugin upgrade-all after applying, then restart."
"note":"Exact third-party payload of the installer bundle. Regenerate with refresh-bundle-lock.ps1 and COMMIT the change as a reviewed dependency bump.",
$problems+="$name/$rel does NOT match the lock (expected sha256 $($expected[$rel].sha256.Substring(0,12))..., got $($actual[$rel].sha256.Substring(0,12))...)"
'The installer installs it from the bundle when -ClientIpSource is direct. Behind a reverse proxy that already sets X-Forwarded-For, use -ClientIpSource proxy and leave it out.'
}
}
Invoke-Check'IIS''Existing sites'{
if(-not$script:IisPresent){
Add-Result'IIS''Existing sites''SKIP''IIS not installed'
Add-Result'MySQL''Backup client''PASS'"mysqldump found ($found)"
}else{
Add-Result'MySQL''Backup client''WARN''mysqldump not found on this server'`
'Needed for the automatic pre-upgrade backup and for "shopdb-admin.ps1 backup". A first install works without it; upgrades will not be protected. It is not on this server yet - the installer stages its own copy, so this normally resolves itself during installation.'
}
}
Invoke-Check'MySQL''Version and config'{
# Find mysqld.exe via the service binary path; read the version and locate my.ini.
# Present in the FILE is not the same as ACTIVE. MySQL must be
# restarted for them to take effect, and the app's own
# `flask db-utils preflight` queries the live server - trust that.
Add-Result'MySQL''5.6 index flags''WARN''all three present in my.ini'`
'Present in the file only. They take effect after a MySQL RESTART, which interrupts the classic ASP app. Confirm with SHOW VARIABLES or flask db-utils preflight.'
}else{
# WARN, not FAIL. This inspects the LOCAL MySQL, which may not be the
# database the operator is about to install against - a bundled 8.4,
# or a remote server. Blocking the wizard here refused an install
# over a server that had nothing to do with it. Stage 3 runs
# 'flask db-utils preflight' against the database actually chosen,
# which is the check that can genuinely block.
Add-Result'MySQL''5.6 index flags''WARN'("missing: "+($missing-join', '))`
"Add to [mysqld] in $ini and restart MySQL, or 'flask db upgrade' fails with error 1071. NOTE: restarting interrupts the classic ASP app."
# One-shot schema + seed. Runs to completion and exits; api waits for it.
# Every step is idempotent, so it is safe to run on every `up`.
migrate:
image:${SHOPDB_IMAGE:?SHOPDB_IMAGE must be set to the channel image ref}
pull_policy:always
restart:"no"
depends_on:
db:
condition:service_healthy
environment:
<<:*app-env
command:
- sh
- -c
- >
flask db upgrade &&
flask plugin upgrade-all &&
flask seed permissions &&
flask seed settings &&
flask seed reference-data
volumes:
- instance_data:/app/instance
api:
image:${SHOPDB_IMAGE:?SHOPDB_IMAGE must be set to the channel image ref}
pull_policy:always
restart:unless-stopped
depends_on:
db:
condition:service_healthy
migrate:
condition:service_completed_successfully
environment:
<<:*app-env
ports:
# Loopback only, same as the DB below. This stack generates its own
# dev secrets and may hold a real imported dataset; it has no business
# listening on every interface of a laptop on a shop-floor network.
- "127.0.0.1:${API_PORT:-5001}:5001"
volumes:
- instance_data:/app/instance
volumes:
db_data:
instance_data:
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.