diff --git a/docs/DEPLOY.md b/docs/DEPLOY.md index 07decaf..c23fcc4 100644 --- a/docs/DEPLOY.md +++ b/docs/DEPLOY.md @@ -7,12 +7,12 @@ shopdb-flask is single-tenant per ADR-004. Each adopting facility runs its own s - Docker 24+ and Docker Compose v2 (or equivalent container runtime) - A reverse proxy with TLS termination (nginx, traefik, Caddy, GE corporate LB) -- the framework does not terminate TLS itself - A MySQL backup destination (offsite recommended) -- Access to the GE Aerospace Gitea or a clone of the repo +- Access to the internal GE Aerospace git server, or a clone of the repo ## Step 1: Clone and configure ```bash -git clone https://gitea.proudtech.net/ge-aerospace/shopdb-flask.git +git clone /ge-aerospace/shopdb-flask.git cd shopdb-flask cp .env.example .env ``` diff --git a/docs/IMPORT-API.md b/docs/IMPORT-API.md index 33d2293..c5d04c2 100644 --- a/docs/IMPORT-API.md +++ b/docs/IMPORT-API.md @@ -497,7 +497,7 @@ need it to remap foreign keys (a machine's `businessunitid`, a checkout's After each phase, compare counts. Legacy side (read-only), for example: ```bash -docker exec dev-mysql mysql -uroot -prootpassword prodscratch \ +docker exec dev-mysql mysql -uroot -p"$MYSQL_ROOT_PASSWORD" prodscratch \ -e "SELECT COUNT(*) FROM vendors;" ``` diff --git a/docs/PROJECT-REVIEW.md b/docs/PROJECT-REVIEW.md index 16e2073..6d9b70b 100644 --- a/docs/PROJECT-REVIEW.md +++ b/docs/PROJECT-REVIEW.md @@ -3,9 +3,9 @@ As of HEAD `ecf4ef6` (2026-07-30), product `__version__ 0.7.0`, contract `__cont ## 1. Executive Summary -**Overall: healthy engineering, drifting focus.** The stated product vision ("plugin system is the product," CLAUDE.md) is delivered: all 7 refactor phases are done, 13 bundled plugins are contract-compliant, the per-plugin migration regime (ADR-008) is live and exercised (geenforce is at `geenforce0002blobs`, proving the post-cutover chain works in anger), and CI enforces naming, contract, and real-MySQL migration idempotency. Test count grew 1077 -> 1159 since the last CLAUDE.md snapshot. +**Overall: healthy engineering, drifting focus.** The stated product vision ("plugin system is the product," the project brief) is delivered: all 7 refactor phases are done, 13 bundled plugins are contract-compliant, the per-plugin migration regime (ADR-008) is live and exercised (geenforce is at `geenforce0002blobs`, proving the post-cutover chain works in anger), and CI enforces naming, contract, and real-MySQL migration idempotency. Test count grew 1077 -> 1159 since the last the project brief snapshot. -The concern is not quality but trajectory. CLAUDE.md and ROADMAP.md both name the last big milestone before 1.0 as "legacy-ASP data import + production pilot." The loader is built and VM-validated (16 stages, `scripts/site_imports/wjf/`), but the prod run has not happened, and ~35 of the last 60 commits went to the GE-Enforce HTTPS cutover instead. That work is legitimate and high-value, but it is feature/fleet work on one plugin, and it has accumulated two process debts that violate the project's own discipline: the cutover playbook (`docs/geenforce-api-cutover.md`) is the only dirty file in the repo and is **untracked**, and the `prod-patch-geenforce` robocopy fast-path can leave prod ahead of git. +The concern is not quality but trajectory. the project brief and ROADMAP.md both name the last big milestone before 1.0 as "legacy-ASP data import + production pilot." The loader is built and VM-validated (16 stages, `scripts/site_imports/wjf/`), but the prod run has not happened, and ~35 of the last 60 commits went to the GE-Enforce HTTPS cutover instead. That work is legitimate and high-value, but it is feature/fleet work on one plugin, and it has accumulated two process debts that violate the project's own discipline: the cutover playbook (`docs/geenforce-api-cutover.md`) is the only dirty file in the repo and is **untracked**, and the `prod-patch-geenforce` robocopy fast-path can leave prod ahead of git. **Verdict: on track against standards, behind against goals.** The 1.0 gate has four items; only one is arguably done, and the roadmap doc does not know it. @@ -13,7 +13,7 @@ The concern is not quality but trajectory. CLAUDE.md and ROADMAP.md both name th | Rule | Status | Evidence | |---|---|---| -| Naming (tables/columns/vars, CONTRIBUTING.md) | **MET** | `scripts/check-naming-and-style.sh` present and executable; dedicated `naming` job in `.gitea/workflows/ci.yml`. One borderline: `asset.py:175` documents a derived API key `location_name` with an underscore - not a DB-mirrored column so likely legal, but worth a glance since "response keys match column names exactly" is the spirit of the rule. | +| Naming (tables/columns/vars, CONTRIBUTING.md) | **MET** | `scripts/check-naming-and-style.sh` present and executable; dedicated `naming` job in the internal CI workflow. One borderline: `asset.py:175` documents a derived API key `location_name` with an underscore - not a DB-mirrored column so likely legal, but worth a glance since "response keys match column names exactly" is the spirit of the rule. | | Plugin contract (manifest.json, BasePlugin, `shopdb.api` only) | **MET** | 13 `plugins/*/manifest.json` verified; contract test suites in `tests/`; contract bumped correctly to 0.15.0 for the geenforce resource-scope fetch tokens (75386d2), per ADR-002 discipline. | | Migration ownership (ADR-008) | **MET** | `PLUGIN_TABLE_OWNERS` registry tested by `tests/test_plugin_migrations.py` (`EXPECTED_HEAD_REVISION` lines 47-51); `migrations-mysql` CI job does fresh utf8mb4 MySQL 8 upgrade + all plugin chains + second-upgrade-no-op assertion. The geenforce `0002blobs` revision shows the per-plugin chain is being used as designed, not just anchored. | | Versioning/release discipline (ADR-007) | **AT RISK** | Tags through v0.7.0 exist and contract bumps are disciplined, but the documentation half of the procedure has drifted - see Gaps 3. | @@ -40,7 +40,7 @@ The concern is not quality but trajectory. CLAUDE.md and ROADMAP.md both name th 1. **Untracked cutover playbook** (`docs/geenforce-api-cutover.md`). The single most valuable in-flight document is one `rm` away from gone, and invisible to any other machine or contributor. 2. **Prod-ahead-of-git debt.** The `prod-patch-geenforce` fast-path means production behavior may not be reproducible from any commit. This directly undermines ADR-012's "engine is source of truth" and the release discipline of ADR-007. -3. **Documentation drift, three concrete instances (all verified):** ROADMAP.md header says contract 0.13.0 (actual 0.15.0); CLAUDE.md says 1077 tests (actual 1159 collected) and claims a "lean-build" CI job that does not exist in `.gitea/workflows/ci.yml` (jobs: backend, naming, frontend, migrations-mysql - lean coverage is folded into pytest via `tests/test_lean_build_guards.py`). Also `.github/workflows/ci.yml` differs from the gitea workflow - one of them is stale. +3. **Documentation drift, three concrete instances (all verified):** ROADMAP.md header says contract 0.13.0 (actual 0.15.0); the project brief says 1077 tests (actual 1159 collected) and claims a "lean-build" CI job that does not exist in the internal CI workflow (jobs: backend, naming, frontend, migrations-mysql - lean coverage is folded into pytest via `tests/test_lean_build_guards.py`). Also `.github/workflows/ci.yml` differs from the internal CI workflow - one of them is stale. 4. **Split-brain fleet enforcement.** Displays/kiosks on the API, the rest of the fleet on the SMB share, with staged-but-unpushed manifest fixes elsewhere (MTConnect v1 stranding). Two delivery mechanisms means two failure modes and doubles the audit surface until the cutover finishes. 5. **1.0 gate criterion 4 unproven:** `docs/DEPLOY.md` has not been validated by an actual fresh-host prod deploy. The air-gapped kit exists; the pilot does not. 6. **ADR-013 limbo:** lean builds shipped under ADR-014 while the catalog/signing model that makes external distribution safe remains PROPOSED. Fine short-term, but gate criterion 3 (external plugin) will force the question. @@ -49,7 +49,7 @@ The concern is not quality but trajectory. CLAUDE.md and ROADMAP.md both name th 1. **Commit `docs/geenforce-api-cutover.md` today.** Zero-cost, eliminates the worst single-point-of-loss risk. 2. **Reconcile prod-patched geenforce files back into git** and gate or retire the robocopy fast-path. Until prod == some tag, ADR-007 is fiction for this plugin. -3. **One doc-sync pass (30 min):** ROADMAP header to 0.15.0, CLAUDE.md test count and CI job list, strike must-have (a) if contract tests confirm the Asset wiring, delete or sync the stale `.github` workflow. +3. **One doc-sync pass (30 min):** ROADMAP header to 0.15.0, the project brief test count and CI job list, strike must-have (a) if contract tests confirm the Asset wiring, delete or sync the stale `.github` workflow. 4. **Finish the cutover or park it cleanly.** Either drive the remaining fleet groups onto the API per section 12, or write down the frozen state as an ADR-012 amendment so the split-brain period is a documented decision, not drift. 5. **Schedule the prodscratch import run and prod pilot.** This is the actual 1.0 milestone and everything is built for it; it validates DEPLOY.md (gate 4) for free. 6. **Pair the equipment one-shot migration with printers retirement** (must-haves b and c) - they are coordinated by design; doing them together avoids touching the legacy tables twice. diff --git a/docs/UPGRADE.md b/docs/UPGRADE.md index 5fd0e89..8fb8399 100644 --- a/docs/UPGRADE.md +++ b/docs/UPGRADE.md @@ -24,7 +24,7 @@ cp -a instance/ instance-backup-$(date +%F)/ git pull origin main ``` -The application is distributed through the internal GE Aerospace Gitea; pull +The application is distributed through the internal GE Aerospace git server; pull from there. There is no external image registry. ## Step 2: Rebuild diff --git a/docs/adr/ADR-007-product-versioning-and-releases.md b/docs/adr/ADR-007-product-versioning-and-releases.md index 1588126..04890b9 100644 --- a/docs/adr/ADR-007-product-versioning-and-releases.md +++ b/docs/adr/ADR-007-product-versioning-and-releases.md @@ -88,8 +88,8 @@ To cut release `X.Y.Z`: ### Neutral -- CI (`.gitea/workflows/ci.yml`) runs the backend tests, the naming/style - gate, and the frontend build on push and PR. It is best-effort: Gitea +- The internal CI workflow runs the backend tests, the naming/style + gate, and the frontend build on push and PR. It is best-effort: the internal CI Actions availability on the host is unverified, so the workflow is config-only until a runner is confirmed. @@ -112,4 +112,4 @@ To cut release `X.Y.Z`: - `shopdb/__init__.py` (`__version__`, `__contract_version__`) - `CHANGELOG.md` (release record) - `frontend/package.json` (frontend version, kept in lock-step) -- `.gitea/workflows/ci.yml` (CI gate) +- the internal CI workflow (CI gate) diff --git a/docs/geenforce-api-cutover.md b/docs/geenforce-api-cutover.md index 56e64ea..75b6b07 100644 --- a/docs/geenforce-api-cutover.md +++ b/docs/geenforce-api-cutover.md @@ -15,7 +15,7 @@ the CUTOVER): - `docs/GE-ENFORCE-CLIENT.md` - client fetch/report contract - `docs/GE-ENFORCE-DEPLOY.md` - what must land on a PC - `docs/GE-ENFORCE-DISPLAY.md` - the display scope specifics -- `/home/camp/projects/pxe/docs/ge-enforce-v2-architecture.md` - the SMB world +- the PXE repo (`docs/ge-enforce-v2-architecture.md`) - the SMB world being cut away from Contents: @@ -410,7 +410,7 @@ A share-less PC cannot pull its first files from the share, so the bootstrap itself is downloadable from the web app. `Install-ShopdbKiosk.ps1` (source of truth: -`/home/camp/pxe-images/shopdb-migration/kiosk-installer/`) is hosted at +the imaging share (`shopdb-migration/kiosk-installer/`)) is hosted at `C:\inetpub\wwwroot\shopdb\installers\kiosk\` and downloadable at `{BaseUrl}/installers/kiosk/Install-ShopdbKiosk.ps1`. Run elevated on the PC: @@ -586,7 +586,7 @@ The VM rig: - The win11 virt-manager VM (see `project-win11-vm` / `reference-vm-qga-as-system` memory), driven by - `/home/camp/pxe-images/ednc-bins/qga-run.py` - qemu guest agent + the imaging share (`ednc-bins/qga-run.py`) - qemu guest agent `guest-exec`, which runs PowerShell AS SYSTEM. That matters: the scheduled tasks run as SYSTEM, so testing as SYSTEM reproduced the session-0 and profile-less behaviors an interactive test would have masked. @@ -612,7 +612,7 @@ git pipeline. Prod is air-gapped from dev; code moves via a git bundle on the share (`\\172.16.9.9\pxe-images\github\shopdb-flask-pub.bundle`) and the .cmd -scripts in `/home/camp/pxe-images/github/`, run on the work PC: +scripts in the imaging share (`github/`), run on the work PC: 1. `pull-shopdb-bundle.cmd` - fetch the bundle into the local clone (ff-only). @@ -634,7 +634,7 @@ the bundle pipeline or prod drifts from git.) ### Channel 2: client + engine + collector + bootstrap (the kiosk bundle) -These live at `/home/camp/pxe-images/shopdb-migration/kiosk-installer/` and +These live at the imaging share (`shopdb-migration/kiosk-installer/`) and deploy by robocopy from the work PC (Z: = share, Y: = prod app dir): ``` @@ -647,7 +647,7 @@ That directory (bundle contents: `Install-ShopdbKiosk.ps1`, the MIME maps) IS the distribution point - PCs download from `{BaseUrl}/installers/kiosk/`. Reference copies of the client kit also live in the repo at `plugins/geenforce/client/` and the engine's source of truth is -`/home/camp/pxe-images/common/lib/Install-FromManifest.ps1`; when the engine +`the imaging share, common/lib/Install-FromManifest.ps1`; when the engine or client changes, update the kiosk bundle copy too (nothing syncs it automatically). PCs pick up new bytes by re-running the bootstrap one-liner. diff --git a/tests/test_docs_publishable.py b/tests/test_docs_publishable.py new file mode 100644 index 0000000..89b97b1 --- /dev/null +++ b/tests/test_docs_publishable.py @@ -0,0 +1,63 @@ +"""docs/ is published to a PUBLIC wiki, so it must not carry internal references. + +The code bundle has a scrub gate in tools/export-github.sh that refuses to commit +when internal names leak. docs/ is EXCLUDED from that bundle - it goes to the +wiki instead, by a generator that has no gate at all. So the one part of the +repository written in prose, by people, about internal infrastructure, was the +one part nothing checked. + +It had leaked: the internal git server's URL, its hostname, `.gitea` workflow +paths, developer home directories, and a dev database root password inside a +copy-pasteable command. + +This test is the gate. It runs in CI, at the source, before anything reaches a +wiki nobody can un-publish. +""" +import re +from pathlib import Path + +import pytest + +REPO = Path(__file__).resolve().parents[1] +DOCS = REPO / 'docs' + +# Kept in step with the scrub list in tools/export-github.sh. Two mechanisms for +# one rule is not ideal, but the export scrubs a tree it is about to commit while +# this one fails a build - and docs/ never reaches the export at all. +FORBIDDEN = [ + (r'gitea', 'names the internal git server'), + (r'proudtech', 'names an internal domain'), + (r'/home/[a-z]+/', 'contains a developer home directory'), + (r'rootpassword', 'contains a database root password'), + (r'\bclaude\b', 'names an LLM assistant'), + (r'\banthropic\b', 'names an LLM vendor'), +] + +# Generated API surface. Not prose, not hand-edited, and regenerated from the +# code by scripts/gen_openapi.py. +SKIP = {'openapi.json', 'api-inventory.json'} + + +def documentation_files(): + return sorted( + path for path in DOCS.rglob('*') + if path.is_file() and path.suffix in {'.md', '.txt'} and path.name not in SKIP + ) + + +def test_there_are_docs_to_check(): + """A path change that silently matched nothing would make this suite pass + while checking absolutely nothing.""" + assert len(documentation_files()) > 20 + + +@pytest.mark.parametrize('pattern,why', FORBIDDEN) +def test_docs_carry_no_internal_references(pattern, why): + offenders = [] + compiled = re.compile(pattern, re.I) + for path in documentation_files(): + for number, line in enumerate(path.read_text(errors='replace').splitlines(), 1): + if compiled.search(line): + offenders.append('%s:%d %s' % (path.relative_to(REPO), number, line.strip()[:100])) + assert not offenders, ( + 'docs/ is published to a public wiki, and this %s:\n %s' % (why, '\n '.join(offenders[:10])))