I asserted that the MySQL migration job "was a comment, not a job" and that
nothing had ever run a fresh install on MySQL. Both wrong. The job existed, hit
this org's ARC runners, and was deliberately moved to internal CI in ffe9fa1cc
because Kubernetes runners without docker-in-docker cannot host service
containers - a fact written at the bottom of the very file I was editing. The
job I added would have failed at container setup on its first push.
Removed it. The leftover was the HEADER, which listed a job that had moved, and
that is now corrected in place with a note not to re-add the entry without
re-adding a job to a runner that can host it. Coverage stays on internal CI.
The api-contract job was green by accident. gen_openapi.py refuses to run while
a bundled plugin is disabled, instance/plugins.json is gitignored, and the step
only worked because the earlier pytest step writes that file at import time and
leaves it. Reordering the steps would have produced a failure about disabled
plugins that had nothing to do with the spec. It now seeds its own registry.
Confirmed against a clean archive of HEAD, where the generator does refuse.
shopdb_whoami claimed to report "what the token is scoped to" and reported the
ACCOUNT's permissions. A read-scoped token on an admin account printed
roles=[admin] and the full permission count, so a model would try a write, take
a 403, and have no tool that explained it. The field is now account_permissions
and carries a note saying a scoped token may allow fewer. Fixing this properly
means /api/auth/me publishing the patscopes claim, which is a separate change.
The MCP client caught only urllib's two error types, so an ordinary mistake
escaped as a raw exception with no cause reaching the caller: a space in a path
raises InvalidURL, a non-ASCII path raises UnicodeEncodeError, a slow read
raises TimeoutError. Paths are now percent-encoded and those escapes are
contained; both cases return the API's own 404 envelope instead.
The operation index collapsed optional authentication. Fourteen routes work
signed out and return more when authenticated, and the index printed them as
requiring a bearer token. They now read bearerAuth,optional.
Also: the generator's own hint told the reader to run "flask plugins enable";
the CLI group is "plugin".
(cherry picked from commit a97346ed81)
The spec described 573 operations with a schema each and almost nothing a
reader could copy: 12 operations carried an example, and no operation described
a query parameter at all, because a Flask handler's query string is invisible to
url_map. A route whose real interface is ?code= documented nothing.
Both are now derived from evidence rather than written by hand.
QUERY PARAMETERS come from each view's AST - request.args.get, the subscript
form, getlist, and this repo's own helpers - and a read that does not resolve to
a literal name is not emitted. 0 to 87 operations, 350 parameter objects. The
enumerations some handlers validate (sort=assetnumber|hostname) are deliberately
NOT published: deriving the accepted set reliably needs more than a literal
read, and a wrong enum is exactly the invented contract this work avoids.
EXAMPLES are harvested from the test suite, which already sends real bodies and
asserts real responses, so they are transcripts rather than inventions. 12 to
about 324. The harvest is re-runnable and optional; generation works without it.
THE HARVEST PUBLISHED TWO REAL CREDENTIALS, and that is the important part of
this commit. A bootstrap token and a kiosk setup code reached docs/openapi.json
inside URL values - a file served at /api/docs on production and copied to a
wiki. Neither scrubber layer could see them: the keys are 'scripturl' and 'url',
which name no secret, and the opaque-value rule excludes ':' and '/' on purpose
so that example URLs survive. Nothing was committed.
Four changes came out of it:
- a URL-credential rule that scrubs the parameter VALUE and keeps the URL, so
?t= is redacted and ?page=2&perpage=50 still reads as an example
- redaction runs on the way OUT as well as on the way in. Harvested examples
live in the inventory and a refresh preserves them, so a scrubber fixed after
a harvest protects only future harvests while the stored copy keeps
publishing. The published file can no longer carry a secret the current rules
would catch, whatever the inventory holds
- the leak gate now holds its own patterns instead of importing the generator's.
Checked with the scrubber's own rules it could only ever assert that redact()
ran, and it was green while both tokens sat in the file. It now knows the
product's token prefixes, a URL-credential shape, and a crude entropy
backstop. Replanting the exact credential that shipped fails it
- a DELETE only declares a body when the HANDLER reads one, derived from its
AST. A transcript proves the client sent something: the suite posts {} to
routes that ignore it, and that published "this route reads a JSON body" on
DELETE /api/endpoints/agentkey, whose handler never calls get_json.
249 operations still carry no example, because no test exercises them with a
JSON body. The honest fix is a test, not a hand-written example, and the next
harvest picks each one up for free.
docs/openapi.json is served live at /api/docs on the production box and
documented none of the routes added since 2026-09-11. It is generated from
docs/api-inventory.json, which is not tracked on this branch, so the generator
had no input - and tests/test_api_inventory_parity.py skipped itself for the
same reason. No route here had ever been checked against the published contract.
The inventory is now generated from this branch's own url_map rather than
carried over from elsewhere, with each route's auth derived from the decorators
and gates on its view rather than guessed. A route whose gate cannot be
classified is marked for review, not quietly called public. gen_openapi.py can
refresh it from the live app, so this does not rot again, and human-authored
prose survives a refresh.
The parity test now RUNS instead of skipping.
The thing that mattered more than the missing pages: nine operations were
published with "security: []" - the machine-readable claim that no credentials
are needed - while their handlers require one. Two of them serve the bootstrap
installer and the agent package. A generated client, a firewall plan or anything
else reading the spec would have been told those are anonymous. Required auth is
now expressed as required, and a token-or-network gate is expressed as optional
auth with the network half named in the description, which is what it actually
is.
A test re-derives auth from the live app and fails when the committed inventory
disagrees, because the parity check only ever compared route lists: a wrong auth
string passed everything.
test_openapi_spec.py asserted that optional auth was the majority of the
surface, a premise the guest-role and permission model has made false. It now
holds the property that matters - an optional-auth route is expressed as
optional - rather than a count that was true once.
gen_openapi.py also loads .env before building the app, like every other script
that builds one, or a plain python run falls back to the hardcoded localhost URL.
Tracking docs/openapi.json (so the app can serve the spec at runtime) makes docs/ exist on the published GitHub mirror, which defeated the skipif(not DOCS.is_dir()) guard in the docs publish/version tests: they then ran against a prose-less checkout and failed on CI. Key the skip off docs/*.md prose instead, matching the canonical published-checkout definition in scripts/check-gates-ran.sh.
Also: the OpenAPI generator emitted a trailing-slash path (/api/docs/), which a strict proxy like IIS treats as a redirecting resource a subpath-mounted client cannot follow; normalise trailing slashes in _norm_path and correct the committed spec. And allow docx (python-docx) as an undeclared import: it is used only by scripts/docs/generate_docx.py, a developer tool that never runs on a deployed site.
