Someone writing a script against ShopDB had two options: read a 705 KB spec, or
guess. The spec is the reference, not an index, and handing an agent the whole
thing means either truncating it or spending a context window on 479 operations
nobody asked about.
Three additions, each feeding the next.
/api/docs/operations.txt is the index: every operation on one line with its
method, path, tag, auth and summary, at around 70 KB for all 486. A caller can
hold the entire API at once, find the four operations it needs, and only then
fetch their schemas.
/api/docs/openapi.json?tag=... narrows the spec to one of the 41 tag groups.
Components are kept intact deliberately: pruning the schema graph to match would
leave references resolving to nothing, and a generator that fails confusingly is
worse than a file that is larger than strictly necessary.
tools/mcp/ is an MCP server built on both. Five tools rather than 483, because a
tool per operation is unusable - clients degrade past a few dozen and models
start picking the wrong one. It reads everything from the live instance at call
time, so enabling a plugin adds operations it can already reach with no change
here. The token is the security boundary and the README says so plainly: every
operation the credential permits is reachable, so exploratory work wants a
read-scoped token.
The rest of this is about the contract not drifting, which is what makes any of
the above trustworthy.
Two CI jobs:
- api-contract runs the parity gates and then regenerates the spec and fails if
anything changed. Those gates already existed and already worked - they caught
both routes added here, unprompted - but nothing ran them except by hand, and
the committed spec had silently drifted by three operations, including two
setup endpoints added the day before.
- migrations-mysql is the job the header comment has been claiming for some
time while not existing. It described "the REAL multi-site deploy path" and
"what catches an Alembic regression on MySQL before it ships", and there was
no such job. That is worse than no comment: it is the false confidence that
let a migration ship which could not build a schema on a fresh MySQL database.
The suite runs on SQLite, and SQLite accepts a foreign key onto a table that
does not exist.
CONTRIBUTING.md now says to run gen_openapi.py --refresh after changing a route,
and that a route the sweep cannot classify belongs in PUBLIC_ENDPOINTS with a
reason rather than hand-edited into the inventory, which the next refresh would
overwrite. The new index route is declared there alongside the spec it indexes.
(cherry picked from commit 762d1b7210)
The spec described 573 operations with a schema each and almost nothing a
reader could copy: 12 operations carried an example, and no operation described
a query parameter at all, because a Flask handler's query string is invisible to
url_map. A route whose real interface is ?code= documented nothing.
Both are now derived from evidence rather than written by hand.
QUERY PARAMETERS come from each view's AST - request.args.get, the subscript
form, getlist, and this repo's own helpers - and a read that does not resolve to
a literal name is not emitted. 0 to 87 operations, 350 parameter objects. The
enumerations some handlers validate (sort=assetnumber|hostname) are deliberately
NOT published: deriving the accepted set reliably needs more than a literal
read, and a wrong enum is exactly the invented contract this work avoids.
EXAMPLES are harvested from the test suite, which already sends real bodies and
asserts real responses, so they are transcripts rather than inventions. 12 to
about 324. The harvest is re-runnable and optional; generation works without it.
THE HARVEST PUBLISHED TWO REAL CREDENTIALS, and that is the important part of
this commit. A bootstrap token and a kiosk setup code reached docs/openapi.json
inside URL values - a file served at /api/docs on production and copied to a
wiki. Neither scrubber layer could see them: the keys are 'scripturl' and 'url',
which name no secret, and the opaque-value rule excludes ':' and '/' on purpose
so that example URLs survive. Nothing was committed.
Four changes came out of it:
- a URL-credential rule that scrubs the parameter VALUE and keeps the URL, so
?t= is redacted and ?page=2&perpage=50 still reads as an example
- redaction runs on the way OUT as well as on the way in. Harvested examples
live in the inventory and a refresh preserves them, so a scrubber fixed after
a harvest protects only future harvests while the stored copy keeps
publishing. The published file can no longer carry a secret the current rules
would catch, whatever the inventory holds
- the leak gate now holds its own patterns instead of importing the generator's.
Checked with the scrubber's own rules it could only ever assert that redact()
ran, and it was green while both tokens sat in the file. It now knows the
product's token prefixes, a URL-credential shape, and a crude entropy
backstop. Replanting the exact credential that shipped fails it
- a DELETE only declares a body when the HANDLER reads one, derived from its
AST. A transcript proves the client sent something: the suite posts {} to
routes that ignore it, and that published "this route reads a JSON body" on
DELETE /api/endpoints/agentkey, whose handler never calls get_json.
249 operations still carry no example, because no test exercises them with a
JSON body. The honest fix is a test, not a hand-written example, and the next
harvest picks each one up for free.
docs/openapi.json is served live at /api/docs on the production box and
documented none of the routes added since 2026-09-11. It is generated from
docs/api-inventory.json, which is not tracked on this branch, so the generator
had no input - and tests/test_api_inventory_parity.py skipped itself for the
same reason. No route here had ever been checked against the published contract.
The inventory is now generated from this branch's own url_map rather than
carried over from elsewhere, with each route's auth derived from the decorators
and gates on its view rather than guessed. A route whose gate cannot be
classified is marked for review, not quietly called public. gen_openapi.py can
refresh it from the live app, so this does not rot again, and human-authored
prose survives a refresh.
The parity test now RUNS instead of skipping.
The thing that mattered more than the missing pages: nine operations were
published with "security: []" - the machine-readable claim that no credentials
are needed - while their handlers require one. Two of them serve the bootstrap
installer and the agent package. A generated client, a firewall plan or anything
else reading the spec would have been told those are anonymous. Required auth is
now expressed as required, and a token-or-network gate is expressed as optional
auth with the network half named in the description, which is what it actually
is.
A test re-derives auth from the live app and fails when the committed inventory
disagrees, because the parity check only ever compared route lists: a wrong auth
string passed everything.
test_openapi_spec.py asserted that optional auth was the majority of the
surface, a premise the guest-role and permission model has made false. It now
holds the property that matters - an optional-auth route is expressed as
optional - rather than a count that was true once.
gen_openapi.py also loads .env before building the app, like every other script
that builds one, or a plain python run falls back to the hardcoded localhost URL.
Document new endpoint that exposes plugin-contributed settings routes
for dynamic frontend UI extension.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Ben Baker's installed-software collection, software delivery (run_script and
update_app), agentless adoption tool and command-queue hardening, with his
authorship preserved.
Reviewed before promotion. Fixed: the endpoints plugin tests were never collected
by pytest, the SBOM snapshot table's key would exhaust on MySQL, ingest did not
truncate two columns strict MySQL would reject, and update_app's version branch
pinned a digest for a file that route never serves. Agent packaging moved off
require_role('admin') onto a new endpoints.agentpackage permission.
Contract 0.27.0. Full suite 2231, frontend 314.
The branch stored scopejson, desiredstate and params as sa.JSON, the only JSON
columns in the product. MySQL 5.6 has no JSON type - it arrived in 5.7 - and the
migration failed there with a bare syntax error near the word JSON.
They are TEXT now, through a JSONText type decorator, so the seventeen call sites
still assign and read dicts. The convention elsewhere is already TEXT:
guides.blocksjson, geenforce.manifestjson.
Spec regenerated: 331 paths, 477 operations across 22 surfaces.
