The full suite caught it: docs/INDEX.md routes by this metadata, so a page
without it is invisible to the router that exists to make the corpus findable.
Failed on gh-main and would have failed CI.
It shipped 1.77 MB of vendored assets to every site - swagger-ui-bundle.min.js
alone is larger than Redoc - in order to provide an Execute button that only the
development config ever enabled. Production config had it off, testing had it
off hard, and when off the routes were never bound at all. The staticdocs
directory drops from 2.9 MB to 1.1 MB, and there is one fewer vendored
dependency to track for licences and advisories.
What replaces it: an agent has the MCP server's shopdb_call, and a human has
Redoc for reading plus the 324 harvested examples in the spec for a curl that is
mostly copy-paste. The read-only page was always the one that mattered.
The tests this leaves behind are the interesting part. That file was named for
try-it but half of it proved things about the page that stays, above all that
nothing it loads comes from the internet, which is what a CDN reference costs on
an air-gapped box. Those are kept and the file is renamed for what it covers.
Pointing the air-gap test at redoc.standalone.js immediately failed: the bundle
still carried a trailing sourceMappingURL for a .map that was never vendored,
so every devtools open was a guaranteed 404 with no internet to fall back on.
That was visible in a browser console and nobody had chased it, because the test
had only ever been parametrised over the two Swagger files. The pragma is gone
and the assertion now checks the TRAILING pragma specifically: Redoc's own code
contains that string three more times, in a data: URI it assembles and in a
worker blob, and a blunt "not in body" assertion fails on correct code.
A first attempt to strip it with a line-anchored pattern removed 507 KB, because
the bundle is minified onto long lines. Hence the anchored, byte-exact removal
and the assertion that can tell a directive from source.
I asserted that the MySQL migration job "was a comment, not a job" and that
nothing had ever run a fresh install on MySQL. Both wrong. The job existed, hit
this org's ARC runners, and was deliberately moved to internal CI in ffe9fa1cc
because Kubernetes runners without docker-in-docker cannot host service
containers - a fact written at the bottom of the very file I was editing. The
job I added would have failed at container setup on its first push.
Removed it. The leftover was the HEADER, which listed a job that had moved, and
that is now corrected in place with a note not to re-add the entry without
re-adding a job to a runner that can host it. Coverage stays on internal CI.
The api-contract job was green by accident. gen_openapi.py refuses to run while
a bundled plugin is disabled, instance/plugins.json is gitignored, and the step
only worked because the earlier pytest step writes that file at import time and
leaves it. Reordering the steps would have produced a failure about disabled
plugins that had nothing to do with the spec. It now seeds its own registry.
Confirmed against a clean archive of HEAD, where the generator does refuse.
shopdb_whoami claimed to report "what the token is scoped to" and reported the
ACCOUNT's permissions. A read-scoped token on an admin account printed
roles=[admin] and the full permission count, so a model would try a write, take
a 403, and have no tool that explained it. The field is now account_permissions
and carries a note saying a scoped token may allow fewer. Fixing this properly
means /api/auth/me publishing the patscopes claim, which is a separate change.
The MCP client caught only urllib's two error types, so an ordinary mistake
escaped as a raw exception with no cause reaching the caller: a space in a path
raises InvalidURL, a non-ASCII path raises UnicodeEncodeError, a slow read
raises TimeoutError. Paths are now percent-encoded and those escapes are
contained; both cases return the API's own 404 envelope instead.
The operation index collapsed optional authentication. Fourteen routes work
signed out and return more when authenticated, and the index printed them as
requiring a bearer token. They now read bearerAuth,optional.
Also: the generator's own hint told the reader to run "flask plugins enable";
the CLI group is "plugin".
Someone writing a script against ShopDB had two options: read a 705 KB spec, or
guess. The spec is the reference, not an index, and handing an agent the whole
thing means either truncating it or spending a context window on 479 operations
nobody asked about.
Three additions, each feeding the next.
/api/docs/operations.txt is the index: every operation on one line with its
method, path, tag, auth and summary, at around 70 KB for all 486. A caller can
hold the entire API at once, find the four operations it needs, and only then
fetch their schemas.
/api/docs/openapi.json?tag=... narrows the spec to one of the 41 tag groups.
Components are kept intact deliberately: pruning the schema graph to match would
leave references resolving to nothing, and a generator that fails confusingly is
worse than a file that is larger than strictly necessary.
tools/mcp/ is an MCP server built on both. Five tools rather than 483, because a
tool per operation is unusable - clients degrade past a few dozen and models
start picking the wrong one. It reads everything from the live instance at call
time, so enabling a plugin adds operations it can already reach with no change
here. The token is the security boundary and the README says so plainly: every
operation the credential permits is reachable, so exploratory work wants a
read-scoped token.
The rest of this is about the contract not drifting, which is what makes any of
the above trustworthy.
Two CI jobs:
- api-contract runs the parity gates and then regenerates the spec and fails if
anything changed. Those gates already existed and already worked - they caught
both routes added here, unprompted - but nothing ran them except by hand, and
the committed spec had silently drifted by three operations, including two
setup endpoints added the day before.
- migrations-mysql is the job the header comment has been claiming for some
time while not existing. It described "the REAL multi-site deploy path" and
"what catches an Alembic regression on MySQL before it ships", and there was
no such job. That is worse than no comment: it is the false confidence that
let a migration ship which could not build a schema on a fresh MySQL database.
The suite runs on SQLite, and SQLite accepts a foreign key onto a table that
does not exist.
CONTRIBUTING.md now says to run gen_openapi.py --refresh after changing a route,
and that a route the sweep cannot classify belongs in PUBLIC_ENDPOINTS with a
reason rather than hand-edited into the inventory, which the next refresh would
overwrite. The new index route is declared there alongside the spec it indexes.
/api/docs was broken in every Docker deployment, including the air-gapped ones.
docs.py resolves openapi.json and llms.txt from the repo root, and the image
copies shopdb, plugins, migrations, scripts and wsgi.py but never docs. The page
rendered the Redoc shell and then answered 404 fetching its own specification.
Only the two files the app actually serves are copied; the rest of that tree is
prose with no business in a runtime image.
The theme was invented colours that matched nothing. It now maps the product's
own tokens, and reads the theme the way the app does: the application stores an
explicit light/dark choice and stamps data-theme, so choosing light on a
dark-desktop kiosk used to give a dark documentation page. The operating system
setting is the fallback, exactly as it is in the app.
Light-mode primary, success and error are a shade darker than the product
tokens, because those tokens are designed as fills and fail contrast as text on
the nested panels where type names and required labels sit. Measured with a
browser harness rather than judged by eye: every text node in both themes now
clears WCAG AA, from an initial 5603 reported failures - most of which were the
harness itself ignoring alpha, which is its own lesson about trusting a number.
Four Redoc elements carry hardcoded colours with no theme key and keep targeted
rules, selected by tag and attribute rather than by generated class hashes that
change on every build.
The build now compiles the Python. Docker never imports the application, so a
syntax error sails through every layer and surfaces as a crash-looping
container; that happened, four restarts, while working on this file.
Redoc's upstream licence file was never vendored although its bundle header
points at it. The placeholder added here says exactly that rather than inventing
the third-party notices it is supposed to carry, and the dead redocly.com badge
is hidden: the licence wants the notice in the distribution, not a rendered
link that a site with no internet cannot follow.
The wizard configured five of the product's 129 settings. A site that finished
it still had no timezone, no authentication, no mail server, no backup
destination and no endpoint agent policy, and nothing told the operator so.
Six steps now: Administrator, Site, Authentication, Features, Mail and Backups,
Review. Everything else moves to a standing setup checklist at
/settings/setupchecklist, because a wizard that asked for all 129 settings would
be worse than one that asked for none.
Two new endpoints back it. /setup/checklist reports twelve areas as
ok/partial/todo/na with a link to the page that fixes each, and /setup/health
reports account counts plus whether a plugin toggle has reached every worker.
Both take care not to lie:
- Several settings are SEEDED non-empty (the shipped logo, a placeholder floor
plan, every identifier field), so "is it set" compares against the shipped
default rather than emptiness.
- plugin.endpoints.sharedkeyallowed DEFAULTS TO ALLOWED when its row is absent,
so an absent row is reported as the fleet-wide key being accepted. Reporting
it as refused told an admin the opposite of the truth about the credential
they are migrating away from.
- A plugin is held at "starting" until several DISTINCT workers report it live.
Counting replies instead let one recycled worker answer four times and declare
a change complete while other workers still served the old routes.
Recommended plugins are declared in each plugin's manifest rather than listed in
the wizard, so adding a plugin cannot leave a stale list in another file.
Also fixed: the logo preview was invisible, because the image borrowed a class
that hardcodes a white background while the shipped logo is white; and signing
in did not re-read setup state, so an admin already browsing was never steered
to the wizard and a hard refresh was the only cure.
Enabling a plugin could not make its routes live. Flask refuses
register_blueprint once the app has served a request, and each gunicorn worker
holds its own app object, so registering in the worker that served the toggle
would leave the others answering 404 for the same URL.
Worse, the window before the restart was a window in which workers disagreed
about reality. Four defects, all of which answered 200 while going wrong:
- Enabling a not-yet-installed plugin ALWAYS 500'd on a running app:
install_plugin ran the migrations, updated the registry, and only then called
register_blueprint and hit Flask's assertion. The toggle half-applied and
reported failure.
- Enabling one plugin could hang the entire site. @jwt_required does SELECT
users, so the request's transaction sits open holding a shared metadata lock;
the plugin's migration runs on its own Alembic connection and needs an
EXCLUSIVE lock on users for a foreign key. Each waits on the other, and MySQL
grants metadata locks in request order, so every later login queued behind the
pending exclusive. Observed twice as a total outage from one plugin toggle.
- Each worker rewrote the WHOLE registry file from a snapshot taken at boot, so
concurrent enables erased each other: seven enables persisted three, with
HTTP 200 for all seven.
- Dependency checks read that same stale snapshot, refusing a plugin whose
dependency another worker had just enabled.
Routes still require the process to restart; what changes is who initiates it.
Each worker fingerprints instance/plugins.json at boot and exits gracefully when
it changes, and the supervisor replaces it. That is the primitive gunicorn's own
--reload uses. Recycles are staggered per worker so the pool rolls rather than
dropping at once, and the exit is scoped to gunicorn: on Windows sys.exit() in a
daemon thread ends the thread rather than the process, so IIS keeps the manual
restart until that path is proven on the Windows VM.
The registry now merges under an exclusive lock and replaces atomically, saves
only the entry being changed, and re-reads before any decision is made from it.
ADR-020 records the decision; ADR-013 is amended where it said the UI must tell
the operator to restart. The restart requirement and the honesty requirement
both stand - only the manual step is removed, and only where it is proven.
A fresh install could not build its schema. Core revision
7d51_endpoint_logcollections creates endpointlogcollections with a foreign
key onto endpointcommands, but PLUGIN_TABLE_OWNERS lists both tables as
owned by the endpoints plugin, and only the plugin's 0002 revision creates
endpointcommands.
Core migrations run before any plugin chain, and a plugin chain only runs
once that plugin is enabled, which cannot happen before the schema exists.
So there is no ordering in which the core revision can succeed on an empty
database. MySQL rejects it with 1824, 'Failed to open the referenced table
endpointcommands'. That is the first command the README gives an adopting
site, on the branch that exists for sites to adopt.
Nothing caught it because the suite runs on sqlite:///:memory: and SQLite
accepts a foreign key onto a table that does not exist. The failure is
MySQL-only and therefore invisible to every test.
Fix:
- endpoints revision 0025 creates the table in the plugin chain, after
0002 has created its foreign key target. Guarded, so it is a no-op on
sites that already applied the core revision.
- 7d51 skips when endpointcommands is absent, which is exactly the fresh
install case 0025 then covers. Its downgrade is guarded to match.
- EXPECTED_HEAD_REVISION records the new endpoints head.
Verified against real MySQL 8.4, not SQLite: on an empty database the
README sequence now completes, 7d51 stamps without creating anything, and
enabling the endpoints plugin builds endpointlogcollections with
endpointlogcollections_ibfk_2 pointing at endpointcommands.
The app has a dark theme. /api/docs was the one page that stayed white when the
rest of the site went dark, which is exactly the page somebody has open beside
the app while they work.
Redoc does not read prefers-color-scheme, and the <redoc spec-url> element it
was using takes no theme at all - so the page now initialises through
Redoc.init, which does. Light keeps the default look; dark names the colours
Redoc otherwise derives from the background and gets wrong: body text, the
sidebar, and the right-hand sample panel, which came out dark-on-dark.
It re-initialises on a live scheme change rather than restyling, because Redoc
bakes its colours into styled-components at init - a CSS override reaches the
chrome and misses the content. The listener has the addListener fallback for the
older MediaQueryList, since a shop-floor kiosk browser is exactly where this
gets read.
Nothing new is loaded: same vendored bundle, same spec, no font, no CDN. A test
pins that, because theming is the kind of change that quietly introduces one.
The try-it page is left light. Swagger UI 5 ships no dark theme, so it would
mean vendoring a community stylesheet that fights upstream on every bump - and
it is a development tool that is off in production anyway.
/api/docs on the production site handed out
https://<host>/api/apitokens for an app that lives at /shopdb. Copy that URL
and it 404s; Swagger's try-it fired at the same wrong address.
The mount is a DEPLOYMENT fact and the spec is a BUILD artifact.
gen_openapi.py writes servers: [{"url": "/"}] because at generation time nobody
knows where the app will be hung, and wsgi.py only moves MOUNT_PATH from
PATH_INFO into SCRIPT_NAME when the process starts. Served as written, the file
was right for a root install and wrong for every subpath one - which is how
production runs.
The endpoint now sets `servers` per request from request.script_root: empty at
root, /shopdb under the subpath method, and correct behind a proxy that sets the
prefix. The paths themselves stay mount-free, because the prefix belongs in the
server entry and duplicating it into each path would double it.
The file is about a megabyte, so it is parsed once and cached by mtime, and each
request gets a shallow copy with only `servers` replaced - two requests arriving
on different mounts cannot race on one dict.
A test pins both cases and the fact that paths stay unprefixed. Planting a wrong
server URL fails it.
Two claims did not survive the lean cut.
The plugin list and the overview both advertised notifications as carrying a
"recognition feed". Recognition, along with training and recertification, splits
one notification into a card per employee and renders an employee photo - all of
which needs the employees plugin, which this image does not bundle, so those
three types are no longer seeded here. A site reading the README would have gone
looking for a feature that is deliberately absent. The plugin line now says
which types need employees and why.
"Fast path (Docker)" also implied a slower path exists. On this branch Docker is
the only path: there is no installer here, and the Distribution section above it
already says so. Renamed to "Standing up a site".
The rest of the README was already right for what main is - it lists the twelve
bundled plugins, explains that applications is core rather than a plugin, and
describes the three compose files without mentioning the Windows installer that
lives on the site line.
The lean cut removed scan-darkwing-portable.yml and scan-darkwing-pr.yml from
main on the reasoning that they name an internal scanner and so are
site-specific tooling rather than product. That reasoning was wrong, and it
showed up as scan-darkwing.cmd failing with
HTTP 422: Workflow does not have 'workflow_dispatch' trigger
because the command dispatches against the DEFAULT branch and the file was no
longer there.
A CI workflow is repository infrastructure, not shipped product. It never
reaches a receiving site, so "does a site need this file" is the wrong question
to ask about it - and main is the branch a site pulls FROM, which makes it the
last place that should lose its scanning. Removing them turned security scanning
off on the default branch silently: nothing failed, no test covered it, and the
only signal was a dispatch error days later.
Both restored unchanged. scan-darkwing-portable.yml carries the
workflow_dispatch trigger the command calls; scan-darkwing-pr.yml runs on
pull_request. Neither references anything the lean cut removed.
Checked the rest of .github/ for the same mistake: the only other difference
from the site line is GE-ENFORCE-COLLECTOR-FIX.html, which is correctly absent
because the geenforce plugin is not bundled here.
GitHub main gained a CONTRIBUTING.md change while the lean cut was being built.
Merged rather than rebased: the lean commit is a 675-file tree replacement, and
replaying it onto a new base rewrites it for no benefit when the two touch no
file in common.
main was 16 days and 259 commits behind the line the product is actually built
on, at contract 0.30.0 against 0.35.0, with no per-PC credential code at all. It
was neither current nor curated - just stale. This brings it forward and, at the
same time, makes it mean something: main is what a receiving site installs, and
nothing else.
WHAT IT CARRIES. Twelve plugins - backups, computers, endpoints, guides,
knowledgebase, machines, network, notifications, printers, slides, tools,
warranty - plus applications, which is manifest-less core and always ships. The
only manifest dependency inside that set is endpoints on computers, and both are
in it, so the closure is self-contained.
WHAT IT DOES NOT. employees, geenforce, measuringtools, printedparts and usb
stay on the site line. So do the Windows installer tree, the VM rehearsal
helpers, the agent tooling config, one site's data loader, the internal audit
and findings records, and the planner spreadsheet. 1,135 tracked files against
1,438.
A REMOVAL THAT BREAKS THE APP IS NOT A REMOVAL, and two nearly shipped:
deploy/site-profile-universal.json still named all seventeen plugins. It is the
file docs/DEPLOY.md tells a site to provision from, so a fresh install would
have failed at `flask plugin apply-profile` with PluginError - the first thing a
receiving site does, broken by the branch meant to serve them. It now names the
twelve that exist, and its comment says the two must agree.
agent/ was never refreshed. Whoever populated this branch copied shopdb/,
plugins/, frontend/ and tests/ from the site line and left the agent tree at its
September state: 22 C# and PowerShell sources missing, including AgentCredentials,
EnrollmentService, AgentHttpFactory and the fleet-alert client, plus one file the
site line had deleted still present. The old tree is internally consistent, so it
compiles, and nothing in CI builds it - every gate was green over a server that
speaks enrollment and per-PC credentials to a bundled agent source that cannot.
Resynced whole.
DECISIONS THIS ENCODES, rather than defers:
- The plugin-authoring lab builds printedparts, whose manifest depends on
employees, so it could not stay. The lab remains on the site line and main
ships no build-along tutorial. That is a real loss for an adopting developer
and is recorded here rather than discovered later.
- Recognition, Training and Recertification notifications are no longer seeded.
Each splits one notification into a card per employee and renders a photo,
which needs a plugin this branch does not bundle. Seeding them would offer an
author a type that cannot resolve anybody. A site that installs employees gets
them from that plugin's own seed.
- Windows/IIS is documented as the legacy path throughout, and the README leads
with Docker, because that is what a receiving site gets.
The API description is regenerated for THIS branch - 347 paths and 483
operations, down from 415 and 573 - so the published contract describes what is
here rather than what the site line serves. docs/CONFIG.md lost the two
measuring-tool label keys the code no longer declares, which two lanes had
decided opposite ways.
Suite: 3124 passed, 6 skipped. Naming and style gates pass. The docs gates run
rather than skip, and every index line count is derived from the file it names.
Ben Baker's installed-software collection, software delivery (run_script and
update_app), agentless adoption tool and command-queue hardening, with his
authorship preserved.
Reviewed before promotion. Fixed: the endpoints plugin tests were never collected
by pytest, the SBOM snapshot table's key would exhaust on MySQL, ingest did not
truncate two columns strict MySQL would reject, and update_app's version branch
pinned a digest for a file that route never serves. Agent packaging moved off
require_role('admin') onto a new endpoints.agentpackage permission.
Contract 0.27.0. Full suite 2231, frontend 314.
The dashboard panel now matches the sidebar's own vocabulary. The export refuses
--dist anywhere but main, because the dist directories are what the prod update
script mirrors onto the server and they are not branch-qualified.
Publication now mirrors the branch and writes one bundle per branch. Dashboard:
guests see what they can reach rather than a blank page, and the estate total
lists every type it counts.
The board was one card per source in a fixed grid: every source got the same
small box whatever it had to say (warranties showed 5 of 50), and a box per
source flattened priority, while the largest type on the page belonged to counts
that need nobody.
Now a severity rollup, one worst-first queue of everything needing a person, and
a row of context beneath. The queue is full width, capped per source so a noisy
check cannot bury a rare one, and names what did not fit with a link to it. Chips
filter the queue by check as well as by severity; both narrow what is already
loaded, so the counts and rows cannot disagree.
Dropped: an 'All assets' tile shown beside its own four addends, a printer strip
carrying the same count a third time, three cards whose job was to display 0, and
a Recent Devices table listing five machines all 'In Use'.
Contract 0.26.0 adds optional 'shortname' on dashboard cards - a title is written
as a heading and ran wider than the hostname it qualified as a row badge.
Also: the supplies card reported 'Low Supplies 0' when Zabbix was unreachable,
which reads as reassurance rather than the truth that nothing could see the
cartridges. It now distinguishes unreachable (says so) from unconfigured (stays
quiet - the site declined the feature) from a real zero.
Full suite 2182 passed, frontend 314 passed.
The triage band answers the common case above the fold, so the full edge list is
reference rather than the first thing to read. Same caret and keyboard handling
as the collapsible panels on the same page.
The card stays: it is the only surface with Add and delete, and it keeps each
edge's exact type where the band deduplicates by asset - on a printer, four rows
collapse to two chips and which machine DEFAULTS to it is lost.
The header count is of RENDERED items, not raw edges: a symmetric Dualpath link
is stored in both directions, so counting edges read 3 above a card showing 2.
Frontend 280 passed.
The machine page opened with a dual-bay banner directly above a hero band saying
'Controlled by'. Both answer 'what am I looking at', so the sibling is now a chip
in the band. The band takes a generic extraitems prop rather than learning what
dualpath is - the pairing is resolved by the machines plugin, not from the
relationships endpoint.
MeasuringToolDetail filtered for a machine counterpart, but the import creates a
controls edge PC -> tool, so a gage's counterpart is a computer. It rendered
nothing on every metrology station that has a PC.
Verified on screen. Full suite 2178 passed, frontend 280 passed.
The machine page now shows the PC that controls it. The endpoint was correct all
along and returned both directions; AssetTriageBand discarded every peer whose
type was not 'machine' and was mounted only on the PC page. The counterpart type,
label and relationship types are now props.
Fixed while adopting it more widely: every band chip linked to /assets/<id>, and
there is no such route. The resolver that AssetRelationships.vue kept privately
is now frontend/src/utils/assetRoute.js, used by both - and its type lookup is
case-insensitive, because seed data capitalises the type names while this site's
database does not.
The legacy import is now safe to resume. The 409-resolve asked for
per_page=10000 against a server that clamps to 100, so it silently searched the
first 100 rows and dropped links past that on every re-run. The crosswalk is now
bound to the database it describes, so one carried across a rebuild is refused
rather than attaching rows to the wrong assets. Source database names are
resolved from the environment, so the loader can be aimed without editing it.
Full suite 2178 passed, frontend 280 passed.
Clears every shipped finding the SBOM scan reported: pyjwt 2.13.0, idna 3.19,
axios 1.20.0, postcss 8.5.26, dompurify 3.4.14, nanoid 3.3.18, follow-redirects
1.16.0, form-data 4.0.6. All in-major bumps inside ranges the manifests already
allowed, so this is a lockfile refresh rather than a migration.
The axios and dompurify floors were raised to match, since the old ones still
nominally admitted the vulnerable range.
The PyJWT allowlist entry was removed rather than kept - it stopped matching once
the upgrade landed.
Verified: full suite 2161 passed, frontend 273 passed, build clean, npm audit
--omit=dev reports 0 vulnerabilities.
The rollup took the highest fix version across a component's advisories, which
is wrong whenever an advisory is patched on more than one release branch. nanoid
is fixed on both 3.3.x and 5.x, and it is transitive under a postcss range of
^3.3.11 that no 5.x release can satisfy - so the recommendation was unreachable
rather than merely large.
Per advisory the target is now the lowest fix above the version in use; per
component, the highest of those. Fixes on older branches are skipped rather than
offered as a downgrade.
scripts/scan_sbom.py checks the generated SBOM against OSV.dev by purl. The SBOM
shipped to every server and nothing read it, so 'are we exposed to this advisory'
was still a person going and looking.
Shipped and build-only findings are both reported, but only shipped components
can fail a build - the highest-severity match in the tree today is against the
test runner, and a gate that fails a release over that gets bypassed and then
catches nothing. Severity is taken from the advisory rather than derived from a
CVSS vector, and an advisory with no recorded word is reported UNKNOWN and never
gates.
scripts/vulnerability-allowlist.json records reviewed advisories with the reason,
the accepter, the date, and what would invalidate the reasoning.
Documented in docs/DEPENDENCY-SECURITY.md.
