It shipped 1.77 MB of vendored assets to every site - swagger-ui-bundle.min.js
alone is larger than Redoc - in order to provide an Execute button that only the
development config ever enabled. Production config had it off, testing had it
off hard, and when off the routes were never bound at all. The staticdocs
directory drops from 2.9 MB to 1.1 MB, and there is one fewer vendored
dependency to track for licences and advisories.
What replaces it: an agent has the MCP server's shopdb_call, and a human has
Redoc for reading plus the 324 harvested examples in the spec for a curl that is
mostly copy-paste. The read-only page was always the one that mattered.
The tests this leaves behind are the interesting part. That file was named for
try-it but half of it proved things about the page that stays, above all that
nothing it loads comes from the internet, which is what a CDN reference costs on
an air-gapped box. Those are kept and the file is renamed for what it covers.
Pointing the air-gap test at redoc.standalone.js immediately failed: the bundle
still carried a trailing sourceMappingURL for a .map that was never vendored,
so every devtools open was a guaranteed 404 with no internet to fall back on.
That was visible in a browser console and nobody had chased it, because the test
had only ever been parametrised over the two Swagger files. The pragma is gone
and the assertion now checks the TRAILING pragma specifically: Redoc's own code
contains that string three more times, in a data: URI it assembles and in a
worker blob, and a blunt "not in body" assertion fails on correct code.
A first attempt to strip it with a line-anchored pattern removed 507 KB, because
the bundle is minified onto long lines. Hence the anchored, byte-exact removal
and the assertion that can tell a directive from source.
I asserted that the MySQL migration job "was a comment, not a job" and that
nothing had ever run a fresh install on MySQL. Both wrong. The job existed, hit
this org's ARC runners, and was deliberately moved to internal CI in ffe9fa1cc
because Kubernetes runners without docker-in-docker cannot host service
containers - a fact written at the bottom of the very file I was editing. The
job I added would have failed at container setup on its first push.
Removed it. The leftover was the HEADER, which listed a job that had moved, and
that is now corrected in place with a note not to re-add the entry without
re-adding a job to a runner that can host it. Coverage stays on internal CI.
The api-contract job was green by accident. gen_openapi.py refuses to run while
a bundled plugin is disabled, instance/plugins.json is gitignored, and the step
only worked because the earlier pytest step writes that file at import time and
leaves it. Reordering the steps would have produced a failure about disabled
plugins that had nothing to do with the spec. It now seeds its own registry.
Confirmed against a clean archive of HEAD, where the generator does refuse.
shopdb_whoami claimed to report "what the token is scoped to" and reported the
ACCOUNT's permissions. A read-scoped token on an admin account printed
roles=[admin] and the full permission count, so a model would try a write, take
a 403, and have no tool that explained it. The field is now account_permissions
and carries a note saying a scoped token may allow fewer. Fixing this properly
means /api/auth/me publishing the patscopes claim, which is a separate change.
The MCP client caught only urllib's two error types, so an ordinary mistake
escaped as a raw exception with no cause reaching the caller: a space in a path
raises InvalidURL, a non-ASCII path raises UnicodeEncodeError, a slow read
raises TimeoutError. Paths are now percent-encoded and those escapes are
contained; both cases return the API's own 404 envelope instead.
The operation index collapsed optional authentication. Fourteen routes work
signed out and return more when authenticated, and the index printed them as
requiring a bearer token. They now read bearerAuth,optional.
Also: the generator's own hint told the reader to run "flask plugins enable";
the CLI group is "plugin".
Someone writing a script against ShopDB had two options: read a 705 KB spec, or
guess. The spec is the reference, not an index, and handing an agent the whole
thing means either truncating it or spending a context window on 479 operations
nobody asked about.
Three additions, each feeding the next.
/api/docs/operations.txt is the index: every operation on one line with its
method, path, tag, auth and summary, at around 70 KB for all 486. A caller can
hold the entire API at once, find the four operations it needs, and only then
fetch their schemas.
/api/docs/openapi.json?tag=... narrows the spec to one of the 41 tag groups.
Components are kept intact deliberately: pruning the schema graph to match would
leave references resolving to nothing, and a generator that fails confusingly is
worse than a file that is larger than strictly necessary.
tools/mcp/ is an MCP server built on both. Five tools rather than 483, because a
tool per operation is unusable - clients degrade past a few dozen and models
start picking the wrong one. It reads everything from the live instance at call
time, so enabling a plugin adds operations it can already reach with no change
here. The token is the security boundary and the README says so plainly: every
operation the credential permits is reachable, so exploratory work wants a
read-scoped token.
The rest of this is about the contract not drifting, which is what makes any of
the above trustworthy.
Two CI jobs:
- api-contract runs the parity gates and then regenerates the spec and fails if
anything changed. Those gates already existed and already worked - they caught
both routes added here, unprompted - but nothing ran them except by hand, and
the committed spec had silently drifted by three operations, including two
setup endpoints added the day before.
- migrations-mysql is the job the header comment has been claiming for some
time while not existing. It described "the REAL multi-site deploy path" and
"what catches an Alembic regression on MySQL before it ships", and there was
no such job. That is worse than no comment: it is the false confidence that
let a migration ship which could not build a schema on a fresh MySQL database.
The suite runs on SQLite, and SQLite accepts a foreign key onto a table that
does not exist.
CONTRIBUTING.md now says to run gen_openapi.py --refresh after changing a route,
and that a route the sweep cannot classify belongs in PUBLIC_ENDPOINTS with a
reason rather than hand-edited into the inventory, which the next refresh would
overwrite. The new index route is declared there alongside the spec it indexes.
/api/docs was broken in every Docker deployment, including the air-gapped ones.
docs.py resolves openapi.json and llms.txt from the repo root, and the image
copies shopdb, plugins, migrations, scripts and wsgi.py but never docs. The page
rendered the Redoc shell and then answered 404 fetching its own specification.
Only the two files the app actually serves are copied; the rest of that tree is
prose with no business in a runtime image.
The theme was invented colours that matched nothing. It now maps the product's
own tokens, and reads the theme the way the app does: the application stores an
explicit light/dark choice and stamps data-theme, so choosing light on a
dark-desktop kiosk used to give a dark documentation page. The operating system
setting is the fallback, exactly as it is in the app.
Light-mode primary, success and error are a shade darker than the product
tokens, because those tokens are designed as fills and fail contrast as text on
the nested panels where type names and required labels sit. Measured with a
browser harness rather than judged by eye: every text node in both themes now
clears WCAG AA, from an initial 5603 reported failures - most of which were the
harness itself ignoring alpha, which is its own lesson about trusting a number.
Four Redoc elements carry hardcoded colours with no theme key and keep targeted
rules, selected by tag and attribute rather than by generated class hashes that
change on every build.
The build now compiles the Python. Docker never imports the application, so a
syntax error sails through every layer and surfaces as a crash-looping
container; that happened, four restarts, while working on this file.
Redoc's upstream licence file was never vendored although its bundle header
points at it. The placeholder added here says exactly that rather than inventing
the third-party notices it is supposed to carry, and the dead redocly.com badge
is hidden: the licence wants the notice in the distribution, not a rendered
link that a site with no internet cannot follow.
The wizard configured five of the product's 129 settings. A site that finished
it still had no timezone, no authentication, no mail server, no backup
destination and no endpoint agent policy, and nothing told the operator so.
Six steps now: Administrator, Site, Authentication, Features, Mail and Backups,
Review. Everything else moves to a standing setup checklist at
/settings/setupchecklist, because a wizard that asked for all 129 settings would
be worse than one that asked for none.
Two new endpoints back it. /setup/checklist reports twelve areas as
ok/partial/todo/na with a link to the page that fixes each, and /setup/health
reports account counts plus whether a plugin toggle has reached every worker.
Both take care not to lie:
- Several settings are SEEDED non-empty (the shipped logo, a placeholder floor
plan, every identifier field), so "is it set" compares against the shipped
default rather than emptiness.
- plugin.endpoints.sharedkeyallowed DEFAULTS TO ALLOWED when its row is absent,
so an absent row is reported as the fleet-wide key being accepted. Reporting
it as refused told an admin the opposite of the truth about the credential
they are migrating away from.
- A plugin is held at "starting" until several DISTINCT workers report it live.
Counting replies instead let one recycled worker answer four times and declare
a change complete while other workers still served the old routes.
Recommended plugins are declared in each plugin's manifest rather than listed in
the wizard, so adding a plugin cannot leave a stale list in another file.
Also fixed: the logo preview was invisible, because the image borrowed a class
that hardcodes a white background while the shipped logo is white; and signing
in did not re-read setup state, so an admin already browsing was never steered
to the wizard and a hard refresh was the only cure.
Enabling a plugin could not make its routes live. Flask refuses
register_blueprint once the app has served a request, and each gunicorn worker
holds its own app object, so registering in the worker that served the toggle
would leave the others answering 404 for the same URL.
Worse, the window before the restart was a window in which workers disagreed
about reality. Four defects, all of which answered 200 while going wrong:
- Enabling a not-yet-installed plugin ALWAYS 500'd on a running app:
install_plugin ran the migrations, updated the registry, and only then called
register_blueprint and hit Flask's assertion. The toggle half-applied and
reported failure.
- Enabling one plugin could hang the entire site. @jwt_required does SELECT
users, so the request's transaction sits open holding a shared metadata lock;
the plugin's migration runs on its own Alembic connection and needs an
EXCLUSIVE lock on users for a foreign key. Each waits on the other, and MySQL
grants metadata locks in request order, so every later login queued behind the
pending exclusive. Observed twice as a total outage from one plugin toggle.
- Each worker rewrote the WHOLE registry file from a snapshot taken at boot, so
concurrent enables erased each other: seven enables persisted three, with
HTTP 200 for all seven.
- Dependency checks read that same stale snapshot, refusing a plugin whose
dependency another worker had just enabled.
Routes still require the process to restart; what changes is who initiates it.
Each worker fingerprints instance/plugins.json at boot and exits gracefully when
it changes, and the supervisor replaces it. That is the primitive gunicorn's own
--reload uses. Recycles are staggered per worker so the pool rolls rather than
dropping at once, and the exit is scoped to gunicorn: on Windows sys.exit() in a
daemon thread ends the thread rather than the process, so IIS keeps the manual
restart until that path is proven on the Windows VM.
The registry now merges under an exclusive lock and replaces atomically, saves
only the entry being changed, and re-reads before any decision is made from it.
ADR-020 records the decision; ADR-013 is amended where it said the UI must tell
the operator to restart. The restart requirement and the honesty requirement
both stand - only the manual step is removed, and only where it is proven.
The app has a dark theme. /api/docs was the one page that stayed white when the
rest of the site went dark, which is exactly the page somebody has open beside
the app while they work.
Redoc does not read prefers-color-scheme, and the <redoc spec-url> element it
was using takes no theme at all - so the page now initialises through
Redoc.init, which does. Light keeps the default look; dark names the colours
Redoc otherwise derives from the background and gets wrong: body text, the
sidebar, and the right-hand sample panel, which came out dark-on-dark.
It re-initialises on a live scheme change rather than restyling, because Redoc
bakes its colours into styled-components at init - a CSS override reaches the
chrome and misses the content. The listener has the addListener fallback for the
older MediaQueryList, since a shop-floor kiosk browser is exactly where this
gets read.
Nothing new is loaded: same vendored bundle, same spec, no font, no CDN. A test
pins that, because theming is the kind of change that quietly introduces one.
The try-it page is left light. Swagger UI 5 ships no dark theme, so it would
mean vendoring a community stylesheet that fights upstream on every bump - and
it is a development tool that is off in production anyway.
/api/docs on the production site handed out
https://<host>/api/apitokens for an app that lives at /shopdb. Copy that URL
and it 404s; Swagger's try-it fired at the same wrong address.
The mount is a DEPLOYMENT fact and the spec is a BUILD artifact.
gen_openapi.py writes servers: [{"url": "/"}] because at generation time nobody
knows where the app will be hung, and wsgi.py only moves MOUNT_PATH from
PATH_INFO into SCRIPT_NAME when the process starts. Served as written, the file
was right for a root install and wrong for every subpath one - which is how
production runs.
The endpoint now sets `servers` per request from request.script_root: empty at
root, /shopdb under the subpath method, and correct behind a proxy that sets the
prefix. The paths themselves stay mount-free, because the prefix belongs in the
server entry and duplicating it into each path would double it.
The file is about a megabyte, so it is parsed once and cached by mtime, and each
request gets a shallow copy with only `servers` replaced - two requests arriving
on different mounts cannot race on one dict.
A test pins both cases and the fact that paths stay unprefixed. Planting a wrong
server URL fails it.
main was 16 days and 259 commits behind the line the product is actually built
on, at contract 0.30.0 against 0.35.0, with no per-PC credential code at all. It
was neither current nor curated - just stale. This brings it forward and, at the
same time, makes it mean something: main is what a receiving site installs, and
nothing else.
WHAT IT CARRIES. Twelve plugins - backups, computers, endpoints, guides,
knowledgebase, machines, network, notifications, printers, slides, tools,
warranty - plus applications, which is manifest-less core and always ships. The
only manifest dependency inside that set is endpoints on computers, and both are
in it, so the closure is self-contained.
WHAT IT DOES NOT. employees, geenforce, measuringtools, printedparts and usb
stay on the site line. So do the Windows installer tree, the VM rehearsal
helpers, the agent tooling config, one site's data loader, the internal audit
and findings records, and the planner spreadsheet. 1,135 tracked files against
1,438.
A REMOVAL THAT BREAKS THE APP IS NOT A REMOVAL, and two nearly shipped:
deploy/site-profile-universal.json still named all seventeen plugins. It is the
file docs/DEPLOY.md tells a site to provision from, so a fresh install would
have failed at `flask plugin apply-profile` with PluginError - the first thing a
receiving site does, broken by the branch meant to serve them. It now names the
twelve that exist, and its comment says the two must agree.
agent/ was never refreshed. Whoever populated this branch copied shopdb/,
plugins/, frontend/ and tests/ from the site line and left the agent tree at its
September state: 22 C# and PowerShell sources missing, including AgentCredentials,
EnrollmentService, AgentHttpFactory and the fleet-alert client, plus one file the
site line had deleted still present. The old tree is internally consistent, so it
compiles, and nothing in CI builds it - every gate was green over a server that
speaks enrollment and per-PC credentials to a bundled agent source that cannot.
Resynced whole.
DECISIONS THIS ENCODES, rather than defers:
- The plugin-authoring lab builds printedparts, whose manifest depends on
employees, so it could not stay. The lab remains on the site line and main
ships no build-along tutorial. That is a real loss for an adopting developer
and is recorded here rather than discovered later.
- Recognition, Training and Recertification notifications are no longer seeded.
Each splits one notification into a card per employee and renders a photo,
which needs a plugin this branch does not bundle. Seeding them would offer an
author a type that cannot resolve anybody. A site that installs employees gets
them from that plugin's own seed.
- Windows/IIS is documented as the legacy path throughout, and the README leads
with Docker, because that is what a receiving site gets.
The API description is regenerated for THIS branch - 347 paths and 483
operations, down from 415 and 573 - so the published contract describes what is
here rather than what the site line serves. docs/CONFIG.md lost the two
measuring-tool label keys the code no longer declares, which two lanes had
decided opposite ways.
Suite: 3124 passed, 6 skipped. Naming and style gates pass. The docs gates run
rather than skip, and every index line count is derived from the file it names.
Ben Baker's installed-software collection, software delivery (run_script and
update_app), agentless adoption tool and command-queue hardening, with his
authorship preserved.
Reviewed before promotion. Fixed: the endpoints plugin tests were never collected
by pytest, the SBOM snapshot table's key would exhaust on MySQL, ingest did not
truncate two columns strict MySQL would reject, and update_app's version branch
pinned a digest for a file that route never serves. Agent packaging moved off
require_role('admin') onto a new endpoints.agentpackage permission.
Contract 0.27.0. Full suite 2231, frontend 314.
Applied on top of af13337 so nothing already on wilm is rewritten.
AUTHORIZATION. The six new policy GETs were @jwt_required(optional=True) with
no permission at all, so an anonymous caller could list configuration and
business policies - gated only by a feature flag that 404s while off. They now
require computers.view. The six policy write routes stacked @jwt_required()
above @require_permission, the same pattern removed from nine endpoints routes
in this very push: the outer decorator answers an anonymous request 401 before
require_permission can consult the guest role, so a grant to guests silently
does nothing. Removed, and the now-unused import with them.
tests/test_core/test_public_read_surface.py failed at af13337 with seven ungated
GETs. It passes now. The seventh, endpoints.claim_next_command, is genuinely
agent-facing - authenticated in the handler by the endpoints collector key,
because a bay has no user session - so it joins PUBLIC_ENDPOINTS with its reason
written down, exactly as report_command_result already does in test_authz.py.
Note which half is public: claiming a queued command is the agent's business,
while ENQUEUEING one is the real authority and stays gated on endpoints.command.
INSTALLER. -BaseUrl and -CollectorKey are now mandatory with no defaults, and
https is enforced unless -AllowInsecureHttp is passed for a local test box. The
defaults were a real hostname (ADR-015) and a live collector key: a key in
source is a key in git history, and that key is also the command channel's kill
switch. Removing it here does not undo the disclosure - it must be rotated on
the server, and purged from af13337 - but it stops the next clone carrying it.
IMPORT. A committed import wrote thousands of rows and left no audit trail; it
now writes one AuditLog row inside the same transaction, so it cannot survive a
rollback or go missing from a commit. The computer-type dropdown asked for
per_page: 200, which the API ignores in favour of perpage and clamps to 100
anyway - it uses a new computersApi.types.listAll(), the pattern already applied
to the other selectors. And the comment claiming the template example rows are
"emitted as commented rows the engine skips" was the opposite of the truth: they
import as live data, the emitted instructions say so, and a test depends on it.
Not fixed here, deliberately: rotating the leaked key, purging it from history,
and the endpoint agent's own hardening (per-host credentials, a protected
process denylist so stop cannot kill security tooling, signing before the
PowerShell runner is enabled, registry ACLs, and the 25-second long poll that
will pin a worker thread per bay).
A command reaching 'succeeded' only proves the agent ran the action, not that it posted a new telemetry snapshot. Reloading panels on command success alone re-rendered stale service/process rows, so a stopped process could still show running right after Refresh Status.
Add a lightweight telemetrymeta probe exposing lastreportedat and gate both the header Refresh Status action and inline row Start/Stop on that stamp advancing past a pre-action baseline. Polling is bounded so an offline or idle agent never hangs the button, and messaging distinguishes a confirmed refresh from a queued one.
Bring the .NET endpoint agent source (service, tray, installer) into the repo so Cameron can merge and test, with an agent/.gitignore keeping bin/obj/publish out. Includes the DncMain launch-path fix, tray single-file host env scrub, and best-effort file logger.
Default fuzzy reconcile threshold to 0 and refuse merges when digit runs differ so numbered labels like Bay 311 never bind to Bay 310. Keep edit forms from clearing saved model fields when category filtering hides untyped rows. Includes in-progress computers policy and endpoints command feature work.
The Add Computer, Add Machine, and Add Printer forms offered the entire model catalog and vendor list regardless of what was being added, so a PC could be pointed at a printer model and every vendor showed up whether or not it made that kind of thing. Scope each form to its own asset category so the selectors only present sensible choices.
Add a ?category= filter to GET /api/models that joins modeltypes and matches on category (Equipment, PC, Network, Printer), and include the model's category in the list response so the forms can distinguish types client-side. Untyped models are excluded, since a null category cannot satisfy the filter.
Each subtype form now loads only its category: PCForm requests category PC (servers, which are Network, stay out); MachineForm requests category Equipment, with the main Model dropdown hiding Controller-type models and the Controller dropdown showing only those; PrinterForm requests category Printer, which also fixes its Model list that previously showed the whole catalog.
Vendor dropdowns are derived from the loaded models: a form only offers vendors that actually have a model in its category. Each dropdown keeps the record's own saved vendor even if it falls outside that set, so editing an existing asset never drops its vendor.
PC records now carry Entra/Azure device identity so imported machines keep a durable link back to their directory object instead of matching on hostname alone. Adds azuredeviceid, azureobjectid, and azuredevicecategory to the Computer model with matching migrations, surfaces them read-only in PCDetail as an Entra Device Info card, and lets PCForm edit the device id directly. The computers plugin registers a machineassetnumber import hook that reconciles PC-to-machine links during bulk import and warns on unresolved or conflicting relationships.
Adds importsources adapters that normalize Azure device-group and Thor PC inventory exports into canonical ShopDB CSVs via hostname alignment, exposes register_import_hook and RowWarning on the shopdb.api contract surface (contract 0.21.0), and refreshes the Import Site Assets page with clearer source instructions. Includes reconcile_locations maintenance script and coverage for the new import paths.
Sites seed inventory from the Asset Collector script or spreadsheet exports, but lookup names drift between sources (Dell vs Dell Inc.), which would otherwise create duplicate vendors/models/types on every import. The importer now normalizes and fuzzy-matches near-miss references onto existing lookup rows within an operator-set similarity threshold, auto-applying the match while reporting it for review so nothing merges silently. Operating systems and asset rows are never reconciled. Adds the dedicated Import Site Assets page with script/template download and a multi-file dry-run/commit workflow.
Assign printers to a MACHINE and let whichever PC controls it install them, with
the observed half kept strictly apart from the assigned half: a collector report
can never write an assignment row. Adoption is explicit. Adds a singular
relationship type so defaultprinter can mean exactly one, client scripts that
converge a bay's queues and correct a drifted one in place, and a wave gate so a
fleet-wide driver rollout cannot stampede its own share.
On the container side: the compose stack never persisted the instance directory,
so a rebuild discarded plugins.json and every upload. It is a volume now, the
image ships instance/ owned by the app user so the mountpoint is writable, and
the database moves to MySQL 8.4 LTS, which the Windows runbook has recommended
since 8.0 went end of life. The image also could not build at all: the frontend
stage never copied the plugin staging script it runs.
Adds a topology migration guide covering IIS to Docker and back.
