ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Generate the facts that go stale, and record the fortnight in the changelog

Browse Source 
CLAUDE.md is read at the start of every session and claimed contract 0.16.0 and
1159 tests while the code was at 0.18.0 and 1567, along with a plugin count and
a chain head that were both wrong. Hand-written facts about a moving codebase
are wrong within a fortnight, and a reader who cannot tell which lines are stale
has to re-derive all of them.

So they are derived. scripts/gen_project_map.py reads the versions, the plugin
inventory, every Alembic chain head, the ADR index with status and the size of
the codebase out of the code, and --check fails a build where the committed map
no longer matches. CLAUDE.md now points at it, leads with the multi-site rule,
and lists the gates to run instead of restating the conventions they enforce.

The changelog's Unreleased section covered 8 of the 46 commits since 0.9.0 and
had no Changed, Fixed or Security sections at all. It now carries the whole
fortnight, including both contract bumps - which had never been recorded even
though ADR-002 makes contract versions their own series.
This commit is contained in:
cproudlock
2026-08-14 13:47:48 -04:00
parent 035419fa51
commit 8f9805d964
4 changed files with 543 additions and 124 deletions
Download Patch File Download Diff File Expand all files Collapse all files

234
CLAUDE.md
View File

@@ -1,145 +1,131 @@
# ShopDB Flask Project
Modern rewrite of the classic-ASP shopdb. Built as a framework so sister GE Aerospace sites can adopt it. Plugin system is the product.
Modern rewrite of the classic-ASP shopdb. Built as a framework so sister GE
Aerospace sites can adopt it. The plugin system is the product.
**This site is not the only site.** Anything that names West Jefferson - a
hostname, a share path, a subnet, a machine-number format, a label prefix - is a
defect unless it is a setting with a neutral default, a site-namespaced
directory, or seed data. That is ADR-015, and it is the rule most often broken
in the places tooling cannot see: PowerShell, the installer, JSON, the imaging
share.
## Facts that move live in the map
`docs/PROJECT-MAP.md` is GENERATED. It carries the product and contract
versions, the plugin inventory, every Alembic chain head, the ADR index with
status, and the size of the codebase. Read it instead of asking; regenerate it
rather than correcting it by hand:
```bash
venv/bin/python scripts/gen_project_map.py # write
venv/bin/python scripts/gen_project_map.py --check # CI form
```
Hand-written version numbers in this file were wrong by two contract releases
and 400 tests before the map existed. Nothing that the generator can derive
should be typed here again.
## Where to look
| question | read |
|---|---|
| what a name may be called | `CONTRIBUTING.md` |
| why the architecture is like that | `docs/adr/` (index in the map) |
| what a plugin may touch | `docs/PLUGIN-HOOKS.md` |
| what the collector sends | `docs/COLLECTOR-INTEGRATION.md`, ADR-006 |
| how a site is deployed | `docs/INSTALL-WINDOWS.md`, `docs/OPERATE-WINDOWS.md` |
| how code reaches GitHub | `tools/export-github.sh` |
## Standards are gates, not prose
Compliance is checked, not remembered. Run these rather than re-reading the
rules:
```bash
bash scripts/check-naming-and-style.sh # naming, style, site literals (ADR-015)
venv/bin/python -m pytest -q # includes the docs publishability gate
venv/bin/python -m pytest tests/test_plugin_migrations.py # chain heads registry
venv/bin/python scripts/gen_project_map.py --check
```
A new plugin table means updating `PLUGIN_TABLE_OWNERS` and
`EXPECTED_HEAD_REVISION` in `tests/test_plugin_migrations.py`.
## Database
- **Active database:** `shopdb_flask` (MySQL, asset-based schema)
- **Legacy database:** `shopdb` (Classic ASP schema, used only for one-time data import via `scripts/import_from_mysql.py`)
- **Connection:** `.env` file. See `.env.example`.
- **Active:** `shopdb_flask` (MySQL, asset-based schema)
- **Legacy:** `shopdb` (classic ASP schema; one-time import only)
- **Connection:** `.env`, see `.env.example`
Architecture decisions live in `docs/adr/`. Read those before making schema or contract changes.
Core migrations run with `flask db upgrade`; every plugin owns its own chain
(ADR-008) and runs with `flask plugin upgrade-all`. **A deploy needs both** -
skipping the second is how a new column reaches production as a 1054 error.
- ADR-001: Asset model is the platform contract (Machine retires) - ACCEPTED
- ADR-002: Plugin contract versioning (semver) - ACCEPTED
- ADR-003: Plugin distribution model (in-tree bundled + filesystem-based external) - ACCEPTED
- ADR-004: Deployment topology (per-site instances, not multi-tenant) - ACCEPTED
- ADR-005: Equipment vs measuringtools plugin scope - ACCEPTED
- ADR-006: Plugin collector contract pattern - ACCEPTED
- ADR-007: Product versioning and releases - ACCEPTED
- ADR-008: Plugin migration ownership (per-plugin chains) - ACCEPTED
- ADR-009: Frontend plugin route gating - ACCEPTED
- ADR-010: Frontend plugin hook contract - ACCEPTED
- ADR-011: Machines rename + modeltypes retyping - ACCEPTED
- ADR-012: GE-Enforce manifest ownership in shopdb - ACCEPTED
- ADR-013: Plugin catalog, curated shelf, and lean per-site builds - PROPOSED
- ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, prune not-installed plugin tables) - ACCEPTED
- ADR-015: Where a site's own data is allowed to live (setting with a neutral default, site-namespaced dir, or seed) - ACCEPTED
- ADR-016: Credential delivery to the fleet (ShopDB resolves targeting, encrypted at rest, dedicated fetch scope) - ACCEPTED
Tests run on SQLite, production is MySQL. Dialect drift is not caught by the
suite: `ORDER BY ... NULLS LAST` parses on SQLite and is rejected by MySQL. Sort
in Python when in doubt.
## Coding convention
`CONTRIBUTING.md` defines naming rules (DB tables, columns, Python, JS, Vue, API). Pre-commit hook at `scripts/check-naming-and-style.sh` enforces them. Read `CONTRIBUTING.md` before naming any new identifier.
## Current state (as of 2026-07-13)
Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) largely complete; the last big milestone is the legacy-data import + a production pilot.
### Phases done
- **Phase 0**: 6 ADRs accepted, naming convention v1, pre-commit style hook
- **Phase 1**: 8 smoke tests + 7 production-config tests, Flask-SQLAlchemy 3 fixtures, uv lockfile, hardened ProductionConfig
- **Phase 2**: contract surface defined (`__contract_version__`, `BasePlugin` hooks, `docs/PLUGIN-HOOKS.md`), 51 contract tests
- **Phase 3**: manifest-first loader, fail-loud/isolate policy, contract-version range checking, auto-register core blueprints, `shopdb.api` namespace, `BasePlugin.get_setting/set_setting` helpers
- **Phase 4**: `flask plugin new <name>` CLI, scaffold templates, 14 canary tests, `docs/PLUGIN-QUICKSTART.md`
- **Phase 5**: ADRs moved to `docs/adr/`, Alembic baseline migration, per-site deploy artifacts (`Dockerfile`, `docker-compose.yml`, `docs/DEPLOY.md`)
### Active state
- 1159 tests, naming/style check green, Gitea Actions CI (backend + naming + frontend build + a lean-build job + a migrations-mysql job that runs the real fresh upgrade on utf8mb4 MySQL 8)
- GE-Enforce HTTPS cutover: the displays/kiosks cohort now fetches manifest + inline payloads entirely over HTTPS (share-less); the `gea-shopfloor-display` scope is authored in code (`plugins/geenforce/seed_display_scope.py`) and published via `seed_display_scope(publish=True)`. Other fleet PC types still enforce from the SMB share and only report. See `docs/geenforce-api-cutover.md`.
- `__contract_version__` at 0.16.0 (0.12.0 mailer, 0.13.0 User/Role, 0.14.0 send_webhook, 0.15.0 authorized_service_token, 0.16.0 get_settings_defaults) (product `__version__` 0.7.0, tags v0.5.0/v0.6.0/v0.7.0 - distinct series, ADR-007)
- 13 bundled plugins all satisfy contract: computers, employees, geenforce, knowledgebase, machines, measuringtools, network, notifications, printedparts, printers, slides, usb, warranty
- Core Alembic chain: baseline `68b3947ae14f` -> head `7d26_settings_description_text` (33 core migrations). Each plugin owns its own chain (ADR-008); deploy runs `flask db upgrade` then `flask plugin upgrade-all`. Reproducible + idempotent from empty (env.py relaxes session sql_mode so the chain runs on strict MySQL 8).
- Lean per-site builds (ADR-013 + ADR-014): `scripts/build-site.sh` (backend) + `SITE_PLUGINS` via `scripts/stage-frontend.mjs` (frontend) ship only chosen plugins; `flask plugin prune-schema` drops non-installed plugins' tables at provisioning. Sidebar nav / settings / Displays all gate on staged routes. Manifest-less `plugins/<name>/frontend/` dirs (e.g. `applications`) are core and always ship.
- Windows sites install from a single air-gapped installer `.exe` built per site from its plugin profile (`deploy/windows/installer/`, built by `build-installer.sh` or `build-installer.ps1`). Operator docs: `docs/INSTALL-WINDOWS.md` + `docs/OPERATE-WINDOWS.md` - these are canonical for a NEW site. `docs/INSTALL-WINDOWS-IIS.md` and `docs/DEPLOY-WINDOWS-IIS.md` are the MANUAL procedure, kept for hand-built servers only. The installer verifies its third-party payload against `bundle-lock.json` and installs wheels with `pip --require-hashes`; every build stages a CycloneDX SBOM (`sbom.cdx.json`) onto the server.
- Legacy import: `docs/IMPORT-API.md` is the schema-agnostic import contract; `docs/IMPORT-ADOPTION.md` + `docs/PILOT-DEPLOY.md` cover adopting a site; `scripts/site_imports/wjf/` is the West Jefferson reference loader (all 16 stages, validated end-to-end including on a Windows + MySQL 8 VM).
- API is migration-complete: an admin PAT + docs/IMPORT-API.md let a script import the whole legacy DB (X-Import-Mode preserves timestamps).
- Pre-1.0 framework; sister sites should pin tight `core_version` ranges until contract reaches 1.0
### Deferred
- Equipment data migration (one-shot script for legacy ASP shopdb -> assets). Per ADR-001, only `category='Equipment' AND machinenumber IS NOT NULL` migrates. Skill `migrating-asset-schema` documents the pattern; the actual one-shot script lives in `scripts/migration/` when run.
- Printers retirement: legacy `PrinterData` model + frontend changes. Coordinated with the equipment data migration.
- (DONE 2026-07-11) `measuringtools` plugin (ADR-005) is built and bundled; docs/PLUGIN-GUIDE.md narrates its construction as the plugin tutorial.
- (DONE) Frontend plugin hook contract (ADR-010): get_settings_cards / get_asset_panels / get_map_overlays / get_asset_presentation shipped; generic renderers for panels/overlays land incrementally.
- (DONE) Per-plugin Alembic chains (ADR-008): every bundled plugin carries its own chain; no plugin uses db.create_all().
- Legacy ASP data import against the renamed schema (unblocked; run via docs/IMPORT-API.md) + a production pilot deployment.
## Quick start
```bash
# Start dev environment
~/start-dev-env.sh
# Activate venv and install deps
cd /home/camp/projects/shopdb-flask
source venv/bin/activate
pip install -r requirements.txt
# Configure environment
cp .env.example .env
# Edit .env with DB credentials, JWT secrets
# Create / update database tables via the Alembic chain
flask db upgrade
# Seed RBAC, default settings, and reference data (all idempotent)
flask seed permissions
flask seed settings
flask seed reference-data
# Restart services
pm2 restart shopdb-flask-api shopdb-flask-ui
```
## Service URLs
- Flask API: http://localhost:5001
- Flask UI: http://localhost:5173
- Legacy ASP (data source for one-time import): http://192.168.122.151:8080
`CONTRIBUTING.md` is authoritative: lowercase concatenated DB columns, snake_case
pure-code identifiers, a closed acronym list, banned shorthand. The pre-commit
hook `scripts/check-naming-and-style.sh` enforces what it can.
## Plugin structure
```
plugins/
<plugin_name>/
__init__.py
plugin.py # BasePlugin implementation
manifest.json # Plugin metadata (name, version, dependencies, api_prefix)
models/
__init__.py # Export all models
<model>.py # SQLAlchemy models
api/
__init__.py
routes.py # Flask Blueprint
services/ # Optional, business logic
schemas/ # Optional, marshmallow schemas
migrations/ # Optional, plugin-specific Alembic migrations
plugins/<name>/
manifest.json # metadata, single source of truth (ADR-002)
plugin.py # BasePlugin implementation
models/ # __init__.py exports every model
api/routes.py # Flask blueprint
migrations/ # this plugin's own Alembic chain
services/ schemas/ frontend/ client/ # as needed
```
Each plugin must have:
- `models/__init__.py` exports all models
- `plugin.py` extends `BasePlugin`
- `manifest.json` with metadata (single source of truth per ADR-002)
- No direct imports from core code (use the contract surface defined in ADR-001)
No plugin imports core directly - everything goes through the `shopdb.api`
contract surface. A manifest-less directory under `plugins/` is core frontend
surface and always ships.
## Key files
- `shopdb/__init__.py` - app factory, blueprint registration
- `shopdb/plugins/base.py` - BasePlugin ABC, PluginMeta dataclass
- `shopdb/plugins/loader.py` - filesystem discovery, dependency-aware loading
- `shopdb/core/api/assets.py` - example of optional plugin imports
- `frontend/src/router/index.js` - frontend routing
- `frontend/src/components/AppSidebar.vue` - navigation menu
- `docs/adr/` - architecture decision records
- `shopdb/__init__.py` - app factory, version constants
- `shopdb/plugins/base.py` - BasePlugin, PluginMeta
- `shopdb/plugins/loader.py` - discovery and dependency-aware loading
- `shopdb/plugins/alembic_template.py` - the env every plugin chain runs through
- `frontend/src/router/index.js`, `frontend/src/components/AppSidebar.vue`
- `tools/export-github.sh` - scrub and publication gate
## Migration notes
## State
- `migrations/DATA_MIGRATION_GUIDE.md` - one-time import from legacy ASP shopdb
- `migrations/MIGRATE_USB_DEVICES_FROM_EQUIPMENT.md` - USB device migration from equipment table
- `migrations/FIX_LOCATIONONLY_EQUIPMENT_TYPES.md` - LocationOnly equipment type fix
- `migrations/PRODUCTION_MIGRATION_GUIDE.md` - production import methods
- `migrations/rename_underscore_columns.sql` - one-time rename of snake_case columns to lowercase concatenated (per CONTRIBUTING.md)
- `migrations/versions/` - the core Alembic chain (baseline `68b3947ae14f` -> head `7d26_settings_description_text`). Run `flask db upgrade` to apply.
Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) largely
complete. The last milestone before 1.0 is unchanged and still open: the
legacy-data import plus a production pilot (`scripts/site_imports/wjf/` is the
reference loader, validated end to end).
Pre-1.0 framework. Sister sites pin tight `core_version` ranges until the
contract reaches 1.0.
### Deferred
- Equipment data migration (legacy ASP -> assets). Per ADR-001 only
`category='Equipment' AND machinenumber IS NOT NULL` migrates.
- Printers retirement: legacy `PrinterData` model, coordinated with the above.
## Quick start
```bash
source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env # DB credentials, JWT secrets
flask db upgrade # core chain
flask plugin upgrade-all # every plugin chain
flask seed permissions && flask seed settings && flask seed reference-data
pm2 restart shopdb-flask-api shopdb-flask-ui
```
Dev API on :5001, dev UI on :5173. `venv/bin/python tools/shot.py /route` logs
in and screenshots a page.