Terminate TLS in gunicorn using a host-generated cert mounted read-only under ./certs, so the browser sees a secure context. Secure-context-only web APIs (navigator.clipboard and others) refuse to run over plain http on a non-localhost origin, which the FQDN-accessed test box hits. Healthcheck now probes https with -k to match the TLS listener. A site swaps in its own cert or fronts this with its own IIS/nginx proxy.
Bumps the agent to 1.4.1. run_script fetches a ShopDB-hosted script by digest and runs it in the script's configured context (LocalSystem or the interactive logged-on user), replacing the local scriptpaths allowlist; user-context runs wait for exit and report the real exit code instead of fire-and-forget. update_app resolves an app-level or version-level install source, fetches the pinned bytes, verifies sha256, and runs silently. Both stage into a fixed Defender-excludable folder (C:\ProgramData\ShopDB\EndpointAgent) instead of the LocalSystem temp path. Adds the DeliveryClaim contract for the claim payload.
Adds the two delivery actions to the endpoints command allowlist, each gated on a new endpoints.deploy permission ON TOP OF endpoints.command so pushing software is a heavier grant than a routine toggle. Enqueue validates update_app appversionid ownership and installer presence before queueing. New collector-authenticated download routes let the agent fetch installer packages and script bodies by digest rather than pre-staging them or sending bodies over the command wire. Wires the scriptsApi client, the siteagent generator routes, and a temporary dev-only Delivery test bench (view/route/nav) for exercising the path while the production UI is built.
AppVersion gains sha256 and silentargs so the endpoint update_app push can pin exact installer bytes and carry per-version silent switches; Application gains installersha256 for the app-level install source. New ShopDB-hosted script library (Script model plus scripts API) backs run_script delivery, with the body fetched by digest on push rather than pre-staged or sent over the command wire. Two idempotent migrations with real downgrades. Script added to the shopdb.api contract surface for plugin use.
The command queue is at-most-one-in-flight, but nothing ever expired a stuck command. When an agent claimed a command (RUNNING) then died before reporting a result - killed, uninstalled, or offline mid-flight - the row stayed in-flight forever and every operator action for that endpoint returned 409 'already in flight'. Reap in-flight rows past a staleness ceiling (15m for RUNNING, 60m for PENDING) to a new STATUS_EXPIRED at enqueue time, inside the same transaction, so the queue self-heals lazily the next time someone queues work. No background job and no schema migration - status is free-text.
A misconfigured BaseUrl (plaintext http:// against the TLS port) made the command long-poll fail at connect in milliseconds. The worker misread that fast failure as a normal idle hold-window close and re-polled with no delay, spinning into a request storm. Gate the idle-close path on the poll having stayed open at least MinIdlePollDuration, so a sub-second transport failure takes the transient backoff instead; add a MinPollInterval floor so no path can exceed a safe request rate. Also reject a non-loopback http:// BaseUrl at startup with a single clear error, turning a silent runtime flood into a fast fail. Loopback http stays valid for local dev.
Several days of endpoint-agent work left DB dumps, sample datasets, captured command output, generated maps, vuln scan output, merge-kit exports, and staged agent publish trees in the tree. None are product source. Ignore them so status shows only real work and they cannot be committed by accident. exports/ stays on disk for reference. agent/.gitignore also reserves dev-bin/ and _stage/ for delivery-build scratch.
Tracking docs/openapi.json (so the app can serve the spec at runtime) makes docs/ exist on the published GitHub mirror, which defeated the skipif(not DOCS.is_dir()) guard in the docs publish/version tests: they then ran against a prose-less checkout and failed on CI. Key the skip off docs/*.md prose instead, matching the canonical published-checkout definition in scripts/check-gates-ran.sh.
Also: the OpenAPI generator emitted a trailing-slash path (/api/docs/), which a strict proxy like IIS treats as a redirecting resource a subpath-mounted client cannot follow; normalise trailing slashes in _norm_path and correct the committed spec. And allow docx (python-docx) as an undeclared import: it is used only by scripts/docs/generate_docx.py, a developer tool that never runs on a deployed site.
The endpoint agent already reports an installed-software list (SBOM), but ingress dropped it: the collector schema had no entry for it and there was nowhere to store it, so operators could not see what is installed on a PC.
Persist it as a per-asset snapshot in a new endpointapplications table, replaced on every report (current inventory is what operators ask for, not install history). Add applications to the collector schema and KNOWN_PAYLOAD_KEYS so ingress stops discarding it, a read route for the raw collection, and a collapsed-by-default Installed software panel to surface it. appid is nullable groundwork for a future Managed Applications link; no matcher is wired yet.
Standalone one-shot enrollment tool that creates a ShopDB PC + Endpoint entry using only the endpoints collector key, reusing the existing collector adopt path. Collects identity, WMI hardware, and installed-software SBOM; requires admin. Server-side packaging route attaches to the existing endpoints blueprint; no core contract change.
The agent now reports installed applications from both Uninstall registry hives (64-bit and WOW6432Node) as {displayname, displayversion, publisher}, mirroring the Assess-Orchestrator applications collection. Registry-only, no Win32_Product/WMIC self-repair and no AppX. The server logs it as an unpersisted key until the SBOM store is built.
The endpoint agent now sends disk, bios, memory, processor, osinfo, and domain telemetry. The server does not persist these yet, so ingest logs any unpersisted payload keys with a compact preview. This makes incoming telemetry observable in the logs before any UI or schema work, so we can confirm the agent is sending it and enhance the UI later.
wj held nothing main lacked - git cherry marks every one of its commits as
already present - but promotion cherry-picks, so the histories diverged and a
fast-forward is not possible. Merging rather than resetting keeps the branch
recoverable and needs no force-push.
Levelling first so wilm's patches apply against current product code.
# Conflicts:
# CONTRIBUTING.md
# frontend/src/views/Dashboard.vue
# tools/wj-status.sh
Asked twice what "All assets" covered, which is the signal a label is
underspecified rather than a reader being slow.
The endpoint already counted every status to build its rollup and then returned
two of them, so a reader could see In Use and had no way to learn where the rest
of the total went. It now returns them all, and the panel lists them under the
total - indented and quieter, because they break the total down rather than
adding to it.
Three things this makes visible that were not guessable:
Retired assets are IN the total. Retired is a status on an active record, not
the same as deactivated, and seven of them sit inside this site's 610.
There is an "Inventory" status. I would have hardcoded In Use, In Repair and
Retired and silently dropped it. The rows are built from whatever the API
returns, sorted by count, so a status a site adds appears rather than
disappearing - which matters for sites adopting with their own seed data.
Rows held back by isactive are in NO status bucket, so they cannot be found by
adding the buckets up. That gap is exactly what makes a dashboard total
disagree with a list page, so the count is reported and captioned.
The caption states the counting policy - active records only, dual-bay pairs
counted once - and the dual-path clause renders only when that site toggle is
on, since asserting it unconditionally would be false at a site that has it off.
Full suite 2182, frontend 314.
The panel had no group label before it was one panel, so grouping it needed a
heading and it got 'Estate' - RMM idiom, and the word the leadership documents
already used for the whole of what a site owns.
Wrong register for this surface. The sidebar already groups exactly these things
under a section named assets, so the dashboard had invented a second word for a
group the app already names, and the panel's own total row says 'All assets' -
the heading and the total disagreed inside one panel. 'Estate' is also
enterprise-IT and largely British idiom; it earns its place in an executive
summary and not on a screen read at 6am.
The scoped classes follow the label, so the markup does not keep a word the UI
stopped using. The documents keep 'estate', where it is doing real work for a
leadership reader.
Scope: product
The bundle became one file per branch, but the frontend dists did not: they are
still frontend-dist-subpath-{ops,shopdb}, fixed paths, and update-prod-server.cmd
robocopies the shopdb one onto the production server with /MIR - a full mirror -
with no idea which branch built it.
So --dist from a site branch would have replaced production's frontend with
unreleased work at the next deploy, silently, and nothing downstream could have
told. Worse than the bundle collision it was sitting beside, because the dist is
what actually lands on the server.
Branch-qualifying the directories would only move the trap to whoever pointed a
deploy at the wrong one. Production runs the product, so the export now refuses
and says which commits to promote instead.
Scope: product
Promotion cherry-picks, which gives the commit a new sha on main. wj-status
compared by sha via main..wj, so after promoting ten commits it still listed all
ten as unpromoted - and would have invited promoting the same work twice.
git cherry compares patch content instead and marks an already-present commit
with a leading '-'. When nothing is genuinely outstanding it now says so, counts
what has already landed, and shows how far the branch is BEHIND main with the
rebase to run. Behind was the state that mattered here and the tool could not
express it at all.
Scope: product
Recorded that wilm no longer existed, on the evidence of the local gitea clone
where it was merged and its ref deleted. That was wrong. It lives on the
corporate GitHub and the other site is developing the .NET endpoint agent on it.
The dev box cannot see that branch - no account there - so its absence locally
was never evidence of anything, and fetch-wilm-branch.cmd on the share exists
precisely to bring it back for review. Says so now, so the next reader does not
draw the same conclusion from the same missing ref.
Scope: product
The work-PC instructions began with 'git checkout <branch>', which fails the
first time a branch goes over because it does not exist in that clone yet -
exactly the moment somebody follows them most carefully. Nothing to check out,
and nothing to merge into either.
Now prints both: fetch then checkout -b then push -u the first time, and the
checkout/fetch/merge/push cycle afterwards.
Scope: product
(cherry picked from commit 1fe788a3ed)
promote-to-main.sh checks out main. The tooling lives only on the site branch,
so that checkout removes tools/promote-to-main.sh while the script is still
running - and bash reads a script lazily, by file offset, so it would carry on
executing whatever bytes happen to land there. While manipulating branches.
Not theoretical: main does not have tools/ right now, so the first real use
would have hit it.
Both versions now copy themselves to a temp file and re-exec from there. The
bash copy unlinks itself immediately, since the open handle keeps working. The
Windows copy has the same hazard for the same reason - cmd.exe re-seeks the
batch file after every command - and two extra traps worth naming: the copy runs
from %TEMP%, so the repository root has to travel with it rather than being
derived from %~dp0, and %errorlevel% inside a parenthesised block is expanded
when the block is PARSED, before the call it is meant to report on.
Scope: product
(cherry picked from commit 6d42110a91)
A guest got a heading and nothing else.
THE STRUCTURAL FAULT WAS MINE
Moving the Estate panel into DashboardCards' rail slot put it behind that
component's root v-if, which was `liveCards.length > 0`. A guest is permitted
zero cards, so the whole component disappeared and took the slot with it. Before
that move the tiles were top-level and always rendered. The wrapper no longer
vanishes with the cards.
TWO LIES THE EMPTY STATE WAS TELLING
"Nothing needs attention" is now shown only to somebody who can see the checks.
A reader permitted none of them is not looking at a healthy fleet, they are
looking at nothing, and saying otherwise is the same fault as reporting Low
Supplies 0 while the monitoring is down. The count of PERMITTED cards is now
tracked separately from the count that currently hold rows.
The Estate panel no longer renders zeros for a caller who cannot read totals.
/api/dashboard/summary needs assets.view, and the failure was swallowed into an
empty object that drew a tidy column of zeros - an estate of nothing, stated
with confidence.
WHAT A GUEST GETS INSTEAD
Where they can go: the same permission-filtered navigation the sidebar uses,
which the server already filters to the guest role for a signed-out visitor. So
it cannot offer a link that dead-ends at the router guard, and this page names no
plugin it had to know about. Verified against the running instance - a guest is
offered Guides and Tech Tools, matching guides.view and tools.view.
ALSO: the measuring tools row
totalassets counts five types - machines, PCs, network, printers AND measuring
tools - but only four had rows, so the total stopped matching the addends above
it wherever gages exist: 610 of visible rows under a stated 729. That is exactly
the fault this panel was built to remove, and it looked right here only because
this database has no measuring tools. Every counted type now has a row.
Frontend 314 passed.
(cherry picked from commit 0f2da9beaf)
The export wrote a single shopdb-flask-pub.bundle whatever it published. That
was fine while only main was ever published, and became a trap the moment it was
not: exporting a site branch silently overwrote the bundle holding main's work,
and the work PC's documented "git fetch <bundle> main" then failed because that
ref was no longer in the file.
It had already happened here - two wj exports overwrote a main bundle from the
same morning. The failure is at least loud on the work PC rather than pushing
the wrong branch, but the main bundle has to be regenerated.
Now shopdb-flask-pub-<branch>.bundle. Two branches cannot share one filename.
Scope: product
(cherry picked from commit 1311dea113)
The branch checkout landed after the rsync, where git checkout --force reverts
the tree to the branch's committed state and discards everything rsync had just
written. The export then found no changes and published nothing, while still
printing a bundle path and looking like it had worked. A silent no-op is the
worst shape for this: the next person believes the export ran.
Scope: product
(cherry picked from commit 06c584240e)
export-github.sh syncs the WORKING TREE, not a branch, and then committed
whatever it found onto the publication repo's main and printed
"git push origin main".
So running it while checked out on a site branch published that site's work AS
THE PRODUCT, to the repository every other site consumes, and said nothing. The
branch model made that reachable in one wrong command, from a checkout that
looks completely normal.
It now reads the current branch, mirrors it in the publication repo (creating it
from main the first time, as in the dev repo), bundles that branch, and prints
work-PC instructions naming it. Publishing anything other than main also prints
a reminder that other sites read main and these commits are not there yet.
The publication checkout uses --force, which is safe only because that tree is a
mirror: the rsync has already replaced it, so there is nothing local to lose. It
would not be safe anywhere else.
Also cleans up the publication repo: origin pointed at a bundle in a scratchpad
from a session that ended long ago, and two orphaned remote-tracking refs
(github/main, wilm/review-fixes) survived the removal of their remotes. None of
it broke the export, which never pushes, but a `git fetch` in there failed with
a confusing path.
Scope: product
(cherry picked from commit 81df9607e7)
The convention existed only in the scripts' comments and in one conversation,
which is not where somebody joining looks. CONTRIBUTING is.
Records the two branches, the Scope trailer, the promote flow, and the point
that most work is product: ADR-015 puts site differences in settings, seed data
or scripts/site_imports/<site>/, so a site branch is about timing rather than
locality.
Also names the failure worth watching. Promoting something site-specific by
accident is caught by the ADR-015 gate on the way into main. The real risk is
product work stranded on a site branch because nobody decided, which is what
turned wilm from a merge into a port.
Scope: product
(cherry picked from commit 1d4ac11572)
The scripts could move a commit to main but nothing said which commits should
go, and that decision does not survive being deferred: months later it is a list
of subject lines and a guess.
So it is recorded at commit time, as a trailer:
Scope: product the rest of the world should have it
Scope: site this site alone, keep it here
tools/wj-status.sh reads them and prints main..wj in three groups, with the
promote command for the ones marked product.
The path fallback is deliberately weak. Under ADR-015 the only site-specific
home in the tree is scripts/site_imports/<site>/ - everything else is meant to
be product, with settings and seed data carrying what differs between sites. So
"touches only site_imports" is the one confident site verdict paths can give,
and everything else is reported UNMARKED rather than guessed into a bucket. A
tool that quietly sorted commits by directory would be wrong often enough to
stop being read.
The unmarked message says which way to lean, because the failure mode here is
not promoting something site-specific by accident - the ADR-015 gate catches
that on the way in. It is product work sitting on wj forever because nobody
decided, which is how the wilm branch became a port rather than a merge.
Scope: product
(cherry picked from commit 0400a58786)
Same two operations as the bash pair, for driving git from Windows rather than
from the development box.
CREDENTIALS ARE HANDLED DIFFERENTLY, ON PURPOSE
The bash version reads a token out of secrets.md and feeds it to git through
GIT_ASKPASS, because that box has no credential store. Windows has one: Git for
Windows installs Git Credential Manager and enables it by default, so `git push`
prompts once and Windows Credential Manager keeps it.
So the token handling is deliberately NOT ported. Copying a token into a script,
an environment variable or a remote URL on a machine that already has a
credential store is strictly worse than the thing it would replace, and the
comment in the file says so, because "make it match the other script" is exactly
the tidying somebody would otherwise do later.
THE PROMOTE SCRIPT REFUSES TO RUN WITHOUT BASH
scripts/check-naming-and-style.sh is a shell script, and it carries the ADR-015
site-literal check - the one gate that makes a site branch survivable rather
than a slow leak into the product. Skipping it on a machine without bash would
defeat the entire point of the script, so it stops and says how to fix it
instead of promoting without it.
CRLF
.gitattributes already pinned *.sh to LF so Git Bash can run them; *.cmd is now
pinned to CRLF for the same class of reason in the other direction. cmd.exe
parses a lone LF inconsistently around labels and goto, so an LF-only batch file
can run its happy path and then jump nowhere on the error path - which in a
promote script is the path that matters.
(cherry picked from commit 49a898fc5a)
wj holds this site's work; main is the product every other site sees and the
publication mirror exports. Two scripts, one for each direction.
tools/git-push.sh pushes a branch without leaving the token behind. The way this
was being done by hand - rewrite remote.origin.url to embed the token, push,
rewrite it back - writes the token into .git/config in plaintext on every push,
and leaves it there if the push fails or the terminal closes mid-way, where
`git remote -v` will happily print it. Putting it in the URL on the command line
is no better: argv is visible in `ps` to every user on the box. The token now
goes in through GIT_ASKPASS, so it never reaches the config or the process list
and there is nothing to clean up when the script dies.
tools/promote-to-main.sh cherry-picks NAMED commits onto main, rather than
merging the branch. Merging takes the site-specific work along with the product
work, which is how a site branch quietly becomes the product; naming the commits
keeps that decision explicit and per change.
It runs main's gates after the pick and before any push, and does not push at
all - it prints what landed and leaves that to a human. The ADR-015 site-literal
check is the one that earns its place: it catches a hostname or share path
riding along in a commit that looked generic, which is the failure that makes a
site branch dangerous rather than merely awkward. It caught this very commit,
which named the site in a comment.
Both refuse a dirty tree. Pushing one pushes the last commit rather than what is
in front of you, and cherry-picking from one mixes uncommitted work into the
promotion. On failure the promote script aborts the pick and returns you to the
branch you started on, with main untouched.
(cherry picked from commit 6d9954c6f9)
The work-PC instructions began with 'git checkout <branch>', which fails the
first time a branch goes over because it does not exist in that clone yet -
exactly the moment somebody follows them most carefully. Nothing to check out,
and nothing to merge into either.
Now prints both: fetch then checkout -b then push -u the first time, and the
checkout/fetch/merge/push cycle afterwards.
Scope: product
promote-to-main.sh checks out main. The tooling lives only on the site branch,
so that checkout removes tools/promote-to-main.sh while the script is still
running - and bash reads a script lazily, by file offset, so it would carry on
executing whatever bytes happen to land there. While manipulating branches.
Not theoretical: main does not have tools/ right now, so the first real use
would have hit it.
Both versions now copy themselves to a temp file and re-exec from there. The
bash copy unlinks itself immediately, since the open handle keeps working. The
Windows copy has the same hazard for the same reason - cmd.exe re-seeks the
batch file after every command - and two extra traps worth naming: the copy runs
from %TEMP%, so the repository root has to travel with it rather than being
derived from %~dp0, and %errorlevel% inside a parenthesised block is expanded
when the block is PARSED, before the call it is meant to report on.
Scope: product
A guest got a heading and nothing else.
THE STRUCTURAL FAULT WAS MINE
Moving the Estate panel into DashboardCards' rail slot put it behind that
component's root v-if, which was `liveCards.length > 0`. A guest is permitted
zero cards, so the whole component disappeared and took the slot with it. Before
that move the tiles were top-level and always rendered. The wrapper no longer
vanishes with the cards.
TWO LIES THE EMPTY STATE WAS TELLING
"Nothing needs attention" is now shown only to somebody who can see the checks.
A reader permitted none of them is not looking at a healthy fleet, they are
looking at nothing, and saying otherwise is the same fault as reporting Low
Supplies 0 while the monitoring is down. The count of PERMITTED cards is now
tracked separately from the count that currently hold rows.
The Estate panel no longer renders zeros for a caller who cannot read totals.
/api/dashboard/summary needs assets.view, and the failure was swallowed into an
empty object that drew a tidy column of zeros - an estate of nothing, stated
with confidence.
WHAT A GUEST GETS INSTEAD
Where they can go: the same permission-filtered navigation the sidebar uses,
which the server already filters to the guest role for a signed-out visitor. So
it cannot offer a link that dead-ends at the router guard, and this page names no
plugin it had to know about. Verified against the running instance - a guest is
offered Guides and Tech Tools, matching guides.view and tools.view.
ALSO: the measuring tools row
totalassets counts five types - machines, PCs, network, printers AND measuring
tools - but only four had rows, so the total stopped matching the addends above
it wherever gages exist: 610 of visible rows under a stated 729. That is exactly
the fault this panel was built to remove, and it looked right here only because
this database has no measuring tools. Every counted type now has a row.
Frontend 314 passed.
The export wrote a single shopdb-flask-pub.bundle whatever it published. That
was fine while only main was ever published, and became a trap the moment it was
not: exporting a site branch silently overwrote the bundle holding main's work,
and the work PC's documented "git fetch <bundle> main" then failed because that
ref was no longer in the file.
It had already happened here - two wj exports overwrote a main bundle from the
same morning. The failure is at least loud on the work PC rather than pushing
the wrong branch, but the main bundle has to be regenerated.
Now shopdb-flask-pub-<branch>.bundle. Two branches cannot share one filename.
Scope: product
The branch checkout landed after the rsync, where git checkout --force reverts
the tree to the branch's committed state and discards everything rsync had just
written. The export then found no changes and published nothing, while still
printing a bundle path and looking like it had worked. A silent no-op is the
worst shape for this: the next person believes the export ran.
Scope: product
export-github.sh syncs the WORKING TREE, not a branch, and then committed
whatever it found onto the publication repo's main and printed
"git push origin main".
So running it while checked out on a site branch published that site's work AS
THE PRODUCT, to the repository every other site consumes, and said nothing. The
branch model made that reachable in one wrong command, from a checkout that
looks completely normal.
It now reads the current branch, mirrors it in the publication repo (creating it
from main the first time, as in the dev repo), bundles that branch, and prints
work-PC instructions naming it. Publishing anything other than main also prints
a reminder that other sites read main and these commits are not there yet.
The publication checkout uses --force, which is safe only because that tree is a
mirror: the rsync has already replaced it, so there is nothing local to lose. It
would not be safe anywhere else.
Also cleans up the publication repo: origin pointed at a bundle in a scratchpad
from a session that ended long ago, and two orphaned remote-tracking refs
(github/main, wilm/review-fixes) survived the removal of their remotes. None of
it broke the export, which never pushes, but a `git fetch` in there failed with
a confusing path.
Scope: product
The convention existed only in the scripts' comments and in one conversation,
which is not where somebody joining looks. CONTRIBUTING is.
Records the two branches, the Scope trailer, the promote flow, and the point
that most work is product: ADR-015 puts site differences in settings, seed data
or scripts/site_imports/<site>/, so a site branch is about timing rather than
locality.
Also names the failure worth watching. Promoting something site-specific by
accident is caught by the ADR-015 gate on the way into main. The real risk is
product work stranded on a site branch because nobody decided, which is what
turned wilm from a merge into a port.
Scope: product
The scripts could move a commit to main but nothing said which commits should
go, and that decision does not survive being deferred: months later it is a list
of subject lines and a guess.
So it is recorded at commit time, as a trailer:
Scope: product the rest of the world should have it
Scope: site this site alone, keep it here
tools/wj-status.sh reads them and prints main..wj in three groups, with the
promote command for the ones marked product.
The path fallback is deliberately weak. Under ADR-015 the only site-specific
home in the tree is scripts/site_imports/<site>/ - everything else is meant to
be product, with settings and seed data carrying what differs between sites. So
"touches only site_imports" is the one confident site verdict paths can give,
and everything else is reported UNMARKED rather than guessed into a bucket. A
tool that quietly sorted commits by directory would be wrong often enough to
stop being read.
The unmarked message says which way to lean, because the failure mode here is
not promoting something site-specific by accident - the ADR-015 gate catches
that on the way in. It is product work sitting on wj forever because nobody
decided, which is how the wilm branch became a port rather than a merge.
Scope: product
Same two operations as the bash pair, for driving git from Windows rather than
from the development box.
CREDENTIALS ARE HANDLED DIFFERENTLY, ON PURPOSE
The bash version reads a token out of secrets.md and feeds it to git through
GIT_ASKPASS, because that box has no credential store. Windows has one: Git for
Windows installs Git Credential Manager and enables it by default, so `git push`
prompts once and Windows Credential Manager keeps it.
So the token handling is deliberately NOT ported. Copying a token into a script,
an environment variable or a remote URL on a machine that already has a
credential store is strictly worse than the thing it would replace, and the
comment in the file says so, because "make it match the other script" is exactly
the tidying somebody would otherwise do later.
THE PROMOTE SCRIPT REFUSES TO RUN WITHOUT BASH
scripts/check-naming-and-style.sh is a shell script, and it carries the ADR-015
site-literal check - the one gate that makes a site branch survivable rather
than a slow leak into the product. Skipping it on a machine without bash would
defeat the entire point of the script, so it stops and says how to fix it
instead of promoting without it.
CRLF
.gitattributes already pinned *.sh to LF so Git Bash can run them; *.cmd is now
pinned to CRLF for the same class of reason in the other direction. cmd.exe
parses a lone LF inconsistently around labels and goto, so an LF-only batch file
can run its happy path and then jump nowhere on the error path - which in a
promote script is the path that matters.
wj holds this site's work; main is the product every other site sees and the
publication mirror exports. Two scripts, one for each direction.
tools/git-push.sh pushes a branch without leaving the token behind. The way this
was being done by hand - rewrite remote.origin.url to embed the token, push,
rewrite it back - writes the token into .git/config in plaintext on every push,
and leaves it there if the push fails or the terminal closes mid-way, where
`git remote -v` will happily print it. Putting it in the URL on the command line
is no better: argv is visible in `ps` to every user on the box. The token now
goes in through GIT_ASKPASS, so it never reaches the config or the process list
and there is nothing to clean up when the script dies.
tools/promote-to-main.sh cherry-picks NAMED commits onto main, rather than
merging the branch. Merging takes the site-specific work along with the product
work, which is how a site branch quietly becomes the product; naming the commits
keeps that decision explicit and per change.
It runs main's gates after the pick and before any push, and does not push at
all - it prints what landed and leaves that to a human. The ADR-015 site-literal
check is the one that earns its place: it catches a hostname or share path
riding along in a commit that looked generic, which is the failure that makes a
site branch dangerous rather than merely awkward. It caught this very commit,
which named the site in a comment.
Both refuse a dirty tree. Pushing one pushes the last commit rather than what is
in front of you, and cherry-picking from one mixes uncommitted work into the
promotion. On failure the promote script aborts the pick and returns you to the
branch you started on, with main untouched.
Splitting "cannot see the cartridges" from "nothing is low" was right, but it
treated two unlike cases as one. isconfigured means enabled AND a URL AND a
token, so its absence means the site did not ask for supply monitoring at all -
and a permanent "Supply monitoring is not configured" line is a card reporting
nothing wrong every day, which is how a board teaches people to stop reading it.
It matters for sites adopting ShopDB without Zabbix, who would otherwise inherit
a warning about a feature they declined.
Not configured now returns a plain empty and the card hides. Only configured and
unanswerable reports itself, because that is something meant to work that is not
working.
Also withdraws an unfounded inference. A dev box has no route to Zabbix by
design, so "not reachable" there is the expected state and says nothing about
what production can see. The value of this change is not that it revealed a
production fault; it is that when production's Zabbix does go down, the board
will say so instead of displaying "Low Supplies 0".
Four tests pin the three cases apart: unreachable reports, unconfigured stays
quiet, and a real zero from a reachable Zabbix is not flagged as blindness. The
fourth holds the envelope's rows, since carrying a reason changed the response
shape. Verified they are not vacuous by restoring the original branch in place -
the unreachable test fails against it.
Full suite 2182 passed.
TWO CONTROLS, ONE BOARD
Severity answers "how bad". It could not answer "what kind", and on a fleet
where eight endpoints go quiet at once, being able to set those aside and look
at the four crashes is the difference between a board and a list.
Chips per contributing check, coloured by its severity, composing with the
severity rollup above them. Both narrow what is already loaded rather than
refetching, so no combination can disagree with the counts. Picking a severity
that excludes the active chip clears the chip: two controls lit above an empty
queue, neither obviously wrong, is worse than losing the narrower one.
A chip counts every row, not the rows the queue had room for. A chip reading 5
above a "3 more" line would be its own small lie.
"LOW SUPPLIES 0" WAS NOT TRUE
Asked what happened to toner replacement, and the answer was worse than the
layout question that prompted it.
_get_low_supplies_data returned the same empty result for three different
situations: Zabbix not configured, Zabbix unreachable, and every cartridge
full. The dashboard strip therefore displayed "Low Supplies 0 / Critical 0"
while the monitoring was dead - and this instance is exactly that case, with
zabbix_enabled true against a URL nothing answers on.
So the strip removed in the previous commit was not information. It was
reassurance, and it read identically whether the printers were fine or nobody
could see them. That is the failure this codebase already names elsewhere: an
empty card and a broken card must not look the same.
The endpoint now distinguishes the three, and returns an envelope rather than a
bare list so a card can say WHY it has no rows. cardVisible shows such a card
whatever its empty rule, because "I cannot tell" is not "nothing to report".
It renders as a notice line, not a queue row: there is no asset to act on, and
burying "the monitor is dead" among the things it failed to find would be the
same mistake in a new place.
Note what this does NOT fix: the Zabbix URL on this instance is a dev
placeholder. The board will now say so instead of implying the cartridges are
full, but somebody still has to point it at a real server - and it is worth
checking what production points at, since nothing would have shown there either.
Frontend 314 passed (13 new), full suite 2178. The response shape changed from
a list to an envelope; no test depended on the old shape.
The board was one card per source in a fixed grid, and both halves of that hurt.
Every source got the same small box whatever it had to say. Warranties showed 5
of 50 behind "and 45 more"; the crash card truncated at 4; a one-row card sat
beside a four-row card and left a hole. Nothing had room, which is the complaint
that started this: not enough space to relay the data.
And a box per source flattened priority. An enforcement failure and a warranty
notice looked equally urgent because each owned a card, while the largest type
on the page belonged to counts that are true every day and need nobody.
WHAT THE PAGE IS NOW
A severity rollup, one worst-first queue of everything that needs a person, and
a row of context beneath it.
The queue merges every critical and warning row and sorts them, so reading order
is work order. It is full width, which is where the room came from. Rows are
capped per source, because fifty expiring warranties must not bury two crashed
PCs, and what did not fit is named per source with a link to it - a bare "and 45
more" that leads nowhere is worse than saying nothing.
The rollup counts ROWS, not cards, and filters the board in place rather than
navigating away, so the counts and the rows cannot disagree.
Severity, position and permission were already declared per card and already
sorted on. The old layout threw that away; this only stops it.
WHAT WENT, AND WHY IT SHOULD NOT COME BACK
Four asset tiles plus an "All assets" tile: machines + PCs + printers + network
IS all assets, so the page showed a total beside its own addends at the same
weight. It is now a line under a rule, where a total belongs.
The printer strip. It carried the printer count a third time - the tile row and
"Total Printers" were already the same number - and "Online" equalled total, so
three tiles held one fact. A genuinely low cartridge already reaches the queue
through the printers-supplies card, which is the moment anyone wants it.
Three cards whose whole job was to display 0.
Recent Devices: the largest block on the page, listing five machines that were
all "In Use" with business unit "TBD". Recency is not a reason to look at
something.
CONTRACT 0.26.0: 'shortname'
Cards may declare a short label for their rows in the merged queue. Their
'title' is written as a heading - "Endpoints with crashes or shutdowns" - and as
a row badge it ran wider than the hostname it qualified, putting the badge
rather than the asset at the start of every line. CSS-truncating the title was
tried first and was worse: four consecutive rows reading "ENDPOINTS WITH CR..."
say nothing at all. Optional and defaults to the title, so a plugin that does
not declare one is unaffected. Additive field, minor bump.
ALSO: "silent 1 days"
A meta suffix is written for the plural case and the card cannot know the value.
One row saying it was a shrug; the merged queue puts every quiet endpoint on one
screen, so it said it five times in a column and read as broken. Now singularised
at exactly 1, on the first word, keeping what follows: "1 event (48h)", "silent 1
day". The test asserting ' ms' survives is what caught the first version turning
it into ' m'.
DashboardRow.vue is extracted because the queue and the context panels render
the same row, and the map tooltip, link fallbacks and chips are too easy to get
subtly different in a second copy.
Verified on screen against live data. Full suite 2178 passed, frontend 301.
The triage band now answers the common case above the fold - what an asset
serves, or what drives it - so the full edge list below is reference rather than
the first thing to read. On a machine showing two relationships, both already
appeared in the band and the banner, and the card was a third copy.
Collapsed by default, using the same caret, roles and keyboard handling as the
collapsible panels on the same page rather than a second, slightly different
one.
The card is NOT redundant with the band and is not removed. It is the only
surface with Add Relationship and the per-row delete, and it keeps each edge's
exact type where the band deduplicates by asset. The printer page is the proof:
four rows there - usesprinter and defaultprinter, for each of two machines -
collapse to two band chips, and which machine DEFAULTS to that printer is a
distinction the band deliberately drops. Fine for triage, wrong for management.
THE COUNT IS OF WHAT IS RENDERED, NOT OF EDGES
A collapsed card that just says "Relationships" cannot be told from an empty
one, so the header carries a count. The first version counted raw edges and was
wrong on the first machine it was shown: a symmetric Dualpath link is stored in
BOTH directions in the legacy shape, so it is two rows in outgoing+incoming and
one "Connected" entry on screen. The badge read 3 above a card that shows 2 when
expanded - exactly the mismatch the badge exists to prevent. It now counts
rendered items.
Add Relationship expands the card before opening the modal, so a new row cannot
land behind a closed caret and read as the add having failed.
Verified on screen: machine reads 2 and matches, printer reads 4 and matches.
Frontend 280 passed.
TWO STRIPS OF CHROME SAYING THE SAME KIND OF THING
The machine page opened with a banner - "Dual-bay machine - sibling bay: 2022" -
directly above a hero whose band said "Controlled by". Both answer "what am I
looking at", and splitting them across a banner and a band cost a strip of
vertical space to say so.
The sibling is now a chip in the band beside the controlling PC. The band takes
an `extraitems` prop rather than learning what dualpath is: the pairing is
resolved by the machines plugin, not from the relationships endpoint, so core
cannot derive it, and a dualpath-shaped prop on a core component would be the
machines plugin leaking upward. A page passes [{label, links:[{text, to}]}] and
the band renders it with its own markup, which also keeps the chips visually
identical to the ones it derives itself.
The label stays "Dual-bay sibling" rather than "Dualpath". The banner spelled
the meaning out because a dualpath pair is ONE physical machine in two bays; a
chip reading "Dualpath" would have dropped that.
THE GAGE PAGE FILTERED FOR A COUNTERPART THAT CANNOT EXIST
MeasuringToolDetail was wired counterparttype="machine" labelled "Used on".
Wrong on both counts. The import synthesizes one measuring-tool asset per
metrology PC and a 'controls' edge PC -> tool (stage_metrology), so a gage's
counterpart is a COMPUTER, exactly as on the machine page. Filtering for a
machine peer meant the band rendered nothing on every gage that has a PC.
It looked merely empty rather than wrong because this dev database holds one
measuring tool with no relationships at all, so there was nothing to be missing.
A hand-held gage genuinely has no such edge and correctly shows nothing; a
metrology station now shows the PC that drives it.
The relationships card stays. It is the only surface with Add Relationship and
the per-row delete, and it lists types the band filters out - the band is
triage, the card is management. It does now repeat what the band shows, which
is worth revisiting separately.
Verified on screen. Frontend 280 passed.
Three defects broke the loader's own documented resume guarantee. All three
were silent: nothing raised, the run finished cheerfully, and fewer rows
arrived than anyone counted.
THE PAGE SIZE WAS CLAMPED, SO THE 409-RESOLVE SEARCHED THE FIRST 100 ROWS
_upsert asked for ?per_page=10000 and read one response. The server clamps any
page size to MAX_PAGE_SIZE, which is 100 - not an error, not a warning, just
100 rows. Anything past that resolved to None, so the id never reached the
crosswalk and the run quietly dropped a model's link on every machine that used
it, or an application's installs.
The target already holds more than 100 models and more than 100 applications,
so this fired on any re-run or resume - exactly the case the loader documents
as safe.
_resolve_existing now tries the exact natural-key filter the API has exposed
all along (?modelnumber= on models, ?appname= on applications, ?vendor= on
vendors) and then pages properly, bounded. An endpoint without such a filter
ignores the unknown parameter and the paged walk finds the row.
stage_metrology repeated the same mistake in its own way: it skipped 409s
entirely, so a resumed run left the tool out of the crosswalk and the Controls
edge unmade - metrology PCs driving nothing. stage_printers already had the
resolve branch; that stage was written later and missed it.
THE CROSSWALK WAS NOT BOUND TO THE DATABASE IT DESCRIBED
idmap.json maps this site's legacy ids to assetids in ONE database, and those
assetids mean nothing anywhere else. Keeping it across a drop-and-rebuild left
warranties and printer assignments attaching to whatever asset now holds that
id: wrong data, no error. The runbook's two "drop and rebuild" steps never
mentioned the file.
The loader now writes a uuid marker into the TARGET's own settings table, so it
dies with the data, and stamps it into the crosswalk. A crosswalk built against
a different database is refused before a row is written. The database name
alone would not do - a rebuild keeps the name. A file written before this guard
is adopted rather than refused, so no existing crosswalk is stranded.
Also adds --idmap, so one checkout can drive more than one target.
THE SOURCE DATABASES ARE NAMED, NOT HARDCODED
'shopdb_src' appeared at 29 call sites, so aiming the loader at the prodscratch
restore meant editing run.py. Those names are now LOGICAL and resolved in
Source.db() from SOURCE_DB_MAIN / SOURCE_DB_USB / SOURCE_DB_EMPLOYEES, with
host, port, user and password overridable too. The call sites are untouched -
resolving centrally avoids missing one, and the literals now read as
documentation of which source a stage draws from. The run prints every source
it redirected, so "it imported nothing" can be told apart from "it read the
wrong database".
This is what stood between the loader and the prodscratch run.
TWO SMALLER ONES
_source_password() raised IndexError rather than its intended message whenever
the DATABASE_URL user is not root, which it will not be on a real server.
main() returned 1 on failure into a caller that discarded it. A run that logged
200 endpoint errors exited 0 and read as a success in a deploy log; it now
exits non-zero.
TESTS AND DOCS
The loader had no tests at all. Sixteen now cover the resolve, the crosswalk
binding and the source aliasing - the logic that can run without a MySQL
source, which is the part that was wrong. Verified they are not vacuous by
running the old algorithm against the same fixture: it returns None where the
new one returns the row.
The README claimed 15 stages "complete" with counts from 2026-07-13 that two
later routing changes invalidated, and the runbook told operators to verify a
fresh import against them. Both corrected, with the three gaps NOT fixed here
recorded plainly: warranties, notifications and knowledgebase POST without a
duplicate check so resuming across them doubles rows; there is no --dry-run or
target guard; and the verify stage counts target assets against source machines,
which is apples to oranges.
Full suite 2178 passed.
THE RECIPROCAL LINKS BUG
A PC's page showed "Serves machine 7803" at the top. The machine's page showed
nothing equivalent, though the same edge was right there in the relationships
card at the bottom. That asymmetry read as missing data and was reported as
such.
It was neither missing data nor an API fault. The endpoint returns both
directions and always did; AssetTriageBand already walked outgoing AND incoming
and deduped, with a comment explaining that production holds reciprocal rows.
It then discarded every peer whose asset type was not 'machine', and hardcoded
the label "Serves machine". So the component could only ever work from the PC
side, and it was mounted only on the PC page.
The counterpart type and label are now props, defaulting to the old values so
the PC page is unchanged. The machine page mounts the band with
counterparttype="computer" and the label "Controlled by".
Eliminated with evidence rather than assumed, because this bug had already been
misdiagnosed twice: the endpoint runs two queries and returns both arrays; the
generic card renders both; that route has no pagination to clamp; and a single
permission gate cannot admit one direction and not the other.
THE RELATIONSHIP TYPE WAS ALSO HARDCODED
Not every asset pair is linked by 'controls'. A machine reaches a printer
through usesprinter and defaultprinter, so the band was blank on the printer
page - the one page where the link is not obvious from the asset itself. The
types are now a prop, and the printer page passes its own.
THE CHIPS WERE DEAD LINKS
Found while adopting the band more widely. It linked to /assets/<assetid>.
There is no /assets/:id route and no catch-all, so every chip in the band was
dead - and this change would have propagated that to four more pages.
AssetRelationships.vue already resolved this correctly and privately. That
function is now frontend/src/utils/assetRoute.js and both components use it, so
a third caller cannot get it wrong again. Two things it gets right that are easy
to miss: the detail routes key on pluginid, not assetid, so passing assetid
opens the wrong record; and the lookup is now case-insensitive, because
csvimport.py seeds 'Computer' and 'Machine' capitalised while this site's
database holds them lowercase - a case-sensitive match produced dead links at
whichever site seeded the other way.
The band now carries pluginid and assettypename on each chip, which the
resolver needs.
TESTS
assetRoute had none and neither did the band. Seven cover the resolver,
including the assetid-vs-pluginid confusion and the seeding-case difference.
On the backend, nothing asserted that the TARGET of a directional edge receives
it in `incoming` - only that rows carry isdirectional. That missing assertion is
this bug's exact shape, so it is now pinned: the endpoint was right all along,
and the next such report should not cost anyone a re-diagnosis of the API.
Verified on screen: the machine page shows "Controlled by G6K76CW3ESF" in the
hero, the PC page is unchanged, and a page with no such link renders no empty
band. Full suite 2162 passed, frontend 280 passed.
docs/PLUGINS.md listed sixteen of the seventeen bundled plugins. The missing one
was endpoints - the newest, and the one another site is most likely to ask
about, since it is the Fleet console and the live panels on a PC detail page.
The row records the two things that are not guessable from the name: it depends
on computers because an endpoint is telemetry ABOUT a PC asset rather than a
second record of it, and its four permissions are split deliberately. Read is
grantable to guests by the .view suffix rule; fleet, command and restart are
not, and the plugin declares no override, so the command channel cannot be
opened to anonymous callers by ticking a box.
Ingest is called out as NOT being a route here - agent reports arrive on the
generic collector route under a service token scoped collector.ingest.endpoints.
Somebody looking for the ingest endpoint in this plugin will not find one.
applications/ stays absent on purpose: it has no manifest and no plugin.py, so
per the manifest-less rule it is core frontend surface that always ships, not a
plugin a site can choose.
requirements.txt carried a hand-written header explaining why the lockfile is
universal and hashed, what each flag buys, and a warning not to hand-edit it
because waitress and tzdata were once added by hand and vanished on the next
compile.
Running `uv pip compile` to upgrade two packages wiped all of it, including the
warning about hand-edits vanishing. The header had been documenting its own
failure mode and still lost.
The explanation now lives in requirements.in, which is the file people edit and
which a compile never rewrites. The header is restored in requirements.txt for
anyone reading that file directly, but it is no longer the only copy.
Also records the --upgrade-package form for moving one package without moving
everything else, and the scan_sbom.py check to run before a release.
scan_sbom.py reported 50 advisory matches against shipped components, 16 of them
at or above HIGH. This clears all of them: the scan now reports 0 shipped
findings and the gate passes.
pyjwt 2.12.1 -> 2.13.0
idna 3.13 -> 3.19
axios 1.13.2 -> 1.20.0 (29 advisories)
postcss 8.5.6 -> 8.5.26
dompurify 3.4.11 -> 3.4.14
nanoid 3.3.11 -> 3.3.18
follow-redirects 1.15.11 -> 1.16.0
form-data 4.0.5 -> 4.0.6
Every one is an in-major bump inside a range the manifests already allowed, so
this is a lockfile refresh rather than a migration. The Python side was compiled
with --upgrade-package for each name so nothing else moved; the diff is two
version lines and their hashes.
nanoid is worth recording: it is transitive under postcss at ^3.3.11, and its
advisories are fixed on both the 3.x and 5.x branches. 3.3.18 is the fix that is
actually reachable.
The declared floors for axios and dompurify were raised to the versions in the
lockfile. They were ^1.6.0 and ^3.4.11, which still nominally admitted the
vulnerable range - npm resolves to the highest match so an install was never
going to pick one, but the manifest should state the minimum safe version rather
than rely on that.
axios 1.20 adds optional proxy-agent support, which pulls https-proxy-agent and
agent-base into the tree. Both are Node-only and neither reaches the browser
bundle; the build was checked for them. The main chunk grew 256kB to 271kB
(85.7kB to 90.5kB gzipped) from the rest of the axios delta.
The PyJWT entry in the allowlist was removed rather than kept. It stopped
matching the moment the upgrade landed, and an allowlist that accumulates
entries for advisories that no longer apply is one nobody rereads. The reasoning
is preserved as the worked example in docs/DEPENDENCY-SECURITY.md.
Verified: full suite 2161 passed, frontend 273 passed, build clean,
npm audit --omit=dev reports 0 vulnerabilities.
The rollup took the highest fix version across a component's advisories. That
is wrong whenever an advisory is patched on more than one release branch.
nanoid is the case that exposed it. Both advisories against it are fixed on two
branches - 3.3.16 and 5.1.16, 3.3.18 and 5.1.6 - and the rollup recommended
5.1.16 with a MAJOR warning. But nanoid is transitive, pulled in by postcss at
`^3.3.11`, a range no 5.x release can satisfy. The recommendation was not merely
a bigger job than necessary, it was unreachable: the actual target is 3.3.18, a
patch inside the existing range.
Per advisory the target is now the lowest published fix above the version in
use; per component it is the highest of those, since one upgrade has to satisfy
every advisory against it. Fixes on older branches are skipped rather than
offered as a downgrade - axios advisories list 0.30.x fixes that would have been
a downgrade from 1.13.2.
This also gives the MAJOR label meaning. It now fires only when no fix exists on
the current branch, rather than whenever some branch happened to be higher.
