The export wrote a single shopdb-flask-pub.bundle whatever it published. That
was fine while only main was ever published, and became a trap the moment it was
not: exporting a site branch silently overwrote the bundle holding main's work,
and the work PC's documented "git fetch <bundle> main" then failed because that
ref was no longer in the file.
It had already happened here - two wj exports overwrote a main bundle from the
same morning. The failure is at least loud on the work PC rather than pushing
the wrong branch, but the main bundle has to be regenerated.
Now shopdb-flask-pub-<branch>.bundle. Two branches cannot share one filename.
Scope: product
(cherry picked from commit 1311dea113)
The branch checkout landed after the rsync, where git checkout --force reverts
the tree to the branch's committed state and discards everything rsync had just
written. The export then found no changes and published nothing, while still
printing a bundle path and looking like it had worked. A silent no-op is the
worst shape for this: the next person believes the export ran.
Scope: product
(cherry picked from commit 06c584240e)
export-github.sh syncs the WORKING TREE, not a branch, and then committed
whatever it found onto the publication repo's main and printed
"git push origin main".
So running it while checked out on a site branch published that site's work AS
THE PRODUCT, to the repository every other site consumes, and said nothing. The
branch model made that reachable in one wrong command, from a checkout that
looks completely normal.
It now reads the current branch, mirrors it in the publication repo (creating it
from main the first time, as in the dev repo), bundles that branch, and prints
work-PC instructions naming it. Publishing anything other than main also prints
a reminder that other sites read main and these commits are not there yet.
The publication checkout uses --force, which is safe only because that tree is a
mirror: the rsync has already replaced it, so there is nothing local to lose. It
would not be safe anywhere else.
Also cleans up the publication repo: origin pointed at a bundle in a scratchpad
from a session that ended long ago, and two orphaned remote-tracking refs
(github/main, wilm/review-fixes) survived the removal of their remotes. None of
it broke the export, which never pushes, but a `git fetch` in there failed with
a confusing path.
Scope: product
(cherry picked from commit 81df9607e7)
The export died with no output and no dists, exit 1, having committed nothing.
grep exits 1 when it matches NOTHING, and under set -euo pipefail that failing
pipeline takes the script with it. Once every internal reference had already
been scrubbed, the gitea grep matched nothing, and the SUCCESS case became the
abort. It printed not one line explaining itself, which is what made it cost an
afternoon: the run looked like it had worked.
All three scrub greps are guarded now. This is the third failure of this shape
in this file, after the two fetch/push tag omissions.
Also excludes a stray .env copy from the sync. A hand-made .env.bak-<date>
reached the publication tree, because rsync walks the WORKING TREE and
.gitignore does not apply. The scrub gate caught it, which is what that gate is
for, and it never reached a commit or a bundle - but a file of live credentials
should not get that far. --include '.env.example' comes first so the example
still publishes.
Both verified by planting a violation: a no-match grep under pipefail, and a
planted .env.bak file that the sync now leaves behind.
Two faults with one cause: docs/ is excluded from publication because the prose
becomes the wiki, and two things under docs/ are not prose.
shopdb/core/api/docs.py reads docs/openapi.json AT RUNTIME to serve /api/docs.
Excluding it meant every site installed from the published repo shipped the
ReDoc page, loaded it, and got {"error":"openapi.json not generated"}. The spec
now publishes - it is a product file that happens to live under docs/. The prose
still goes to the wiki and nowhere else, and docs/api-inventory.json stays out,
because nothing reads it at runtime.
The second fault is the reason this was worth chasing. Three gate modules skip
themselves when the file they read is absent, which is correct in a published
checkout and silently wrong on GitHub, which runs against exactly that checkout.
The modules skipped, CI reported green, and an outside branch added nine
undocumented routes and passed. A gate that stops running is worse than no gate,
because nobody looks at it again.
scripts/check-gates-ran.sh asserts each gate ran, with two lists: the ones that
must run everywhere, and the ones needing files publication deliberately leaves
behind - required here, allowed to be absent there. A `gates` job runs it in CI.
Proven in both directions: green in each checkout, and red when the file a gate
reads is removed.
Also purge instance/plugins.json from the publication tree. It was never
published - it is untracked there - but an old copy had sat in that tree for
months, protected from --delete by its own exclude, and it made the published
repo fail its own suite locally: 13 plugins enabled where the code expects 16.
A publication tree that cannot pass its own tests cannot be verified before
shipping. That is the exact trap the comment at the top of the script warns
about, which had claimed a second victim.
Also: concurrency on the workflow, so a second push supersedes the first rather
than queueing behind it holding a self-hosted runner.
The scanner has been reporting the same count for weeks, which is what a rule
that only prints becomes. It now FAILS the build, and it looks where the leaks
actually were: PowerShell, the installer, the seeds, generated JSON, the
frontend - case-insensitively, across plugins, shopdb, scripts, deploy, tools.
A line that is deliberate declares itself with an ADR-015-OK marker and a
reason, so the claim is visible in review instead of tolerated in silence.
What it found, fixed here:
- The shadow client wrote one site's ShopDB URL into HKLM whenever the registry
disagreed. At the site it was written for that reads as healing drift;
anywhere else it overwrites the site's own address on every enforce cycle,
and the site cannot win because the cycle repeats. The bay's value now wins,
an explicit -BaseUrl seeds it, and with neither there is nothing honest to
write, so it says so and skips.
- The kiosk dispatcher fell back to one plant's host when HKLM was unset, so a
kiosk elsewhere quietly opened a server it has no business reaching. The
fallback is now this site's site_base_url, baked in at seed time, and the
dispatcher refuses rather than guessing when neither is set. Its legacy
shortcut matcher derives the host from that URL instead of naming one.
- The OpenAPI generator hardcoded a production hostname into every spec it
generated, which then published to a public wiki. The relative mount is the
only server it can honestly name; a site passes its own by environment.
- Placeholders and examples in the UI and the client help offered real internal
subnets and a real production URL. They now use documentation ranges.
Both publication gates - the export scrub and the docs publishability test -
carry the site patterns, which neither did. One plant's hostname, FQDN and
internal networks are out of the documentation and the generated specs.
Comments naming the reference site are reworded rather than deleted: the
reasoning is worth keeping, the plant name is not what makes it true.
The publication rsync excluded 'tools' to keep the repo-root tools/ dir
off GitHub. rsync patterns without a leading slash match at any depth, so
naming a plugin 'tools' meant plugins/tools/ was dropped too. It was
committed, exported, bundled and deployed, and the first sign of trouble
was `plugin install tools` on the server failing on a manifest.json that
had never been shipped.
The root-intended excludes are anchored, and the sync now verifies that
every plugins/*/manifest.json in the working tree came out the other side.
A silent omission of a whole plugin should not be something a deploy
discovers for us.
rsync --exclude also PROTECTS a path from --delete, so anything that reached the
publication tree before its exclude existed stayed there permanently - invisible
to the sync and surfacing only as a scrub-gate failure. That cost two rounds of
'add the exclude, still fails' on the installer bundle and again on
.pytest_cache. The generated paths are now purged before the sync, so adding an
exclude is sufficient on its own.
tests/test_docs_publishable.py assembles its search terms from fragments: a file
containing the literal strings the scrub greps for tripped that scrub on itself.
Excluding the file from publication would have removed the check from the
repository it protects.
The sync walks the working tree rather than git, so deploy/windows/installer/
bundle came through despite being gitignored - about 100MB of build output
containing a copy of the whole application tree, the wheels and the vendor
installers. Its copies of config.py and requirements.txt then tripped the scrub
gate, which is the only reason it was noticed.
Excluded along with the installer's other generated files. Note that rsync
--exclude also protects a path from --delete, so a copy already in the
publication tree has to be removed by hand once.
The MCP server README names Claude Desktop / Claude Code (it is an MCP
server for those clients), which trips the scrub gate. It is a standalone
local tool distributed via pxe-images/mcp/ + setup-mcp.cmd, not part of the
shipped product, so exclude it from the public repo like docs/ and tools/.
Two instances run on the box (dev /ops + prod /shopdb); each needs its own
base-path build. Build both on every --dist so prod never ships a stale
frontend. tools/ is excluded from publication, so this is dev-tooling only.
Cross-plugin FK blocker ADR-013 cited is already resolved: the FKs into
machines were held only by dead legacy tables (machinerelationships,
printerdata, installedapps, communications.machineid) that existing
migrations 7a01/7c01 already drop. No live plugin table hard-FKs another
plugin. Schema-lean is unblocked.
Enabling change: create_plugin_tables now skips already-existing tables
(idempotent) so a plugin anchor can create its tables on a fresh lean
install and no-op on a database that has them from the pre-cutover
baseline. The load-bearing baseline lift is staged as ADR-014 Phase 2.
Loader: bool() on pymysql bit(1) bytes is always true - isinstallable
and isshopfloor imported as 1 for every row; route through _truthy_bit.
The employee source DB is now optional (shopdb-only imports).
Frontend: under a subpath mount the 401 interceptor stored the browser
path (mount base included) as the login redirect and the router applied
its base again (/ops/ops). New stripBase() keeps redirects base-free.
tools/export-github.sh automates the publication flow: prune + scrub +
commit into ~/projects/shopdb-flask-pub and emit a transfer bundle.
