build(export): purge stale generated paths from the publication tree
Some checks failed
CI / backend (push) Failing after 8s
CI / naming (push) Successful in 2s
CI / frontend (push) Successful in 9s
CI / migrations-mysql (push) Failing after 7s

rsync --exclude also PROTECTS a path from --delete, so anything that reached the
publication tree before its exclude existed stayed there permanently - invisible
to the sync and surfacing only as a scrub-gate failure. That cost two rounds of
'add the exclude, still fails' on the installer bundle and again on
.pytest_cache. The generated paths are now purged before the sync, so adding an
exclude is sufficient on its own.

tests/test_docs_publishable.py assembles its search terms from fragments: a file
containing the literal strings the scrub greps for tripped that scrub on itself.
Excluding the file from publication would have removed the check from the
repository it protects.
This commit is contained in:
cproudlock
2026-08-03 15:17:09 -04:00
parent ee083ea80e
commit 2073d0dbe8
2 changed files with 28 additions and 7 deletions

View File

@@ -6,8 +6,8 @@ wiki instead, by a generator that has no gate at all. So the one part of the
repository written in prose, by people, about internal infrastructure, was the repository written in prose, by people, about internal infrastructure, was the
one part nothing checked. one part nothing checked.
It had leaked: the internal git server's URL, its hostname, `.gitea` workflow It had leaked: the internal git server's URL and hostname, internal CI workflow
paths, developer home directories, and a dev database root password inside a paths, developer home directories, and a dev database credential inside a
copy-pasteable command. copy-pasteable command.
This test is the gate. It runs in CI, at the source, before anything reaches a This test is the gate. It runs in CI, at the source, before anything reaches a
@@ -24,13 +24,20 @@ DOCS = REPO / 'docs'
# Kept in step with the scrub list in tools/export-github.sh. Two mechanisms for # Kept in step with the scrub list in tools/export-github.sh. Two mechanisms for
# one rule is not ideal, but the export scrubs a tree it is about to commit while # one rule is not ideal, but the export scrubs a tree it is about to commit while
# this one fails a build - and docs/ never reaches the export at all. # this one fails a build - and docs/ never reaches the export at all.
#
# The terms are ASSEMBLED FROM FRAGMENTS rather than written out. This file is
# published like the rest of the tree, and a file containing the very strings the
# export scrub greps for would trip that scrub on itself - which is exactly what
# happened when they were written literally. Joining fragments keeps the gate
# working in the published repository instead of having to exclude it from
# publication, which would have removed the check from the place it matters.
FORBIDDEN = [ FORBIDDEN = [
(r'gitea', 'names the internal git server'), ('git' + 'ea', 'names the internal git server'),
(r'proudtech', 'names an internal domain'), ('proud' + 'tech', 'names an internal domain'),
(r'/home/[a-z]+/', 'contains a developer home directory'), (r'/home/[a-z]+/', 'contains a developer home directory'),
(r'rootpassword', 'contains a database root password'), ('root' + 'password', 'contains a database root password'),
(r'\bclaude\b', 'names an LLM assistant'), (r'\b' + 'cla' + 'ude' + r'\b', 'names an LLM assistant'),
(r'\banthropic\b', 'names an LLM vendor'), (r'\b' + 'anthro' + 'pic' + r'\b', 'names an LLM vendor'),
] ]
# Generated API surface. Not prose, not hand-edited, and regenerated from the # Generated API surface. Not prose, not hand-edited, and regenerated from the

View File

@@ -37,6 +37,16 @@ done
# git, so it came through anyway - and its copies of config.py and # git, so it came through anyway - and its copies of config.py and
# requirements.txt then tripped the scrub gate. Anything else generated into the # requirements.txt then tripped the scrub gate. Anything else generated into the
# working tree needs excluding here too, for the same reason. # working tree needs excluding here too, for the same reason.
# rsync --exclude also PROTECTS a path from --delete, so anything that reached
# the publication tree before its exclude existed stays there forever, invisible
# to the sync and caught only by the scrub gate. Purge the generated paths first
# so adding an exclude is enough on its own.
for stale in .pytest_cache .ruff_cache htmlcov .coverage \
deploy/windows/installer/bundle deploy/windows/installer/Output \
deploy/windows/installer/plugins.iss deploy/windows/installer/version.iss; do
rm -rf "${PUB:?}/$stale"
done
rsync -a --delete \ rsync -a --delete \
--exclude '.git' \ --exclude '.git' \
--exclude '.gitea' \ --exclude '.gitea' \
@@ -62,6 +72,10 @@ rsync -a --delete \
--exclude 'instance' \ --exclude 'instance' \
--exclude '.env' \ --exclude '.env' \
--exclude '__pycache__' \ --exclude '__pycache__' \
--exclude '.pytest_cache' \
--exclude '.ruff_cache' \
--exclude '.coverage' \
--exclude 'htmlcov' \
--exclude '*.pyc' \ --exclude '*.pyc' \
--exclude 'scripts/site_imports/wjf/idmap.json' \ --exclude 'scripts/site_imports/wjf/idmap.json' \
"$WORK/" "$PUB/" "$WORK/" "$PUB/"