ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
bf75cdc43bfff5cce1d7a4db74a726a71e3b83fa
shopdb-flask/CLAUDE.md
cproudlock 91143d94fb
Some checks failed
CI / backend (push) Failing after 8s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details
ADR-016: credential delivery to the fleet 
Controller and share credentials are currently edited as cleartext into a
script on the SFLD share, one value for the whole fleet, re-applied by
GE-Enforce every cycle. Rotating means editing that file and hoping - nothing
reports which bays picked the new value up - and sites need per-bay variation,
which a manifest scope cannot express because scopes are per PC TYPE. A scope
per bay is a spreadsheet, not a model.

ShopDB therefore owns the definitions and the targeting, and a PC receives a
decided answer rather than rules to evaluate. Targeting uses the axes
GE-Enforce already filters on plus those the asset model knows: hostname
(including -like globs), machine, model, controller OS, PC type with alias
expansion, GE-Enforce profile, and a selector over the stored DNC projection -
"every machine whose DataHost is that share" is answerable from data already
held, and it also answers which machines need a credential at all.

Overlapping matches are guaranteed rather than exceptional, so precedence is
explicit: an integer priority, then scope specificity, then rule id for
stability. The resolver must be able to explain which rule won and why, or
nobody will trust it. It reuses plugins/geenforce/filters.py; there are already
two implementations of this matching logic kept honest by a parity harness, and
a third would drift silently.

Targeting stays separate from detection, mirroring PCTypes versus
DetectionMethod: who gets a credential is a server question, whether it is
already applied is a local one, and conflating them rewrites the secret every
cycle instead of healing drift.

Stored Fernet-encrypted with the key OUTSIDE the database, write-only through
the API and masked on read, fetched with a dedicated credentials.fetch scope so
a leaked collector key does not yield controller passwords, and every fetch
audited.

PROPOSED, not accepted: it makes ShopDB a credential store, and the key becomes
a single point of loss - restore the database without it and every credential
is unrecoverable, which docs/BACKUP-RESTORE.md must state in the same change
that implements this. Registry-only provisioning is recommended as a first step
regardless, since it removes the cleartext from the share immediately and the
client helper is identical either way.
2026-08-11 10:54:42 -04:00

146 lines
9.3 KiB
Markdown
Raw Blame History

# ShopDB Flask Project
Modern rewrite of the classic-ASP shopdb. Built as a framework so sister GE Aerospace sites can adopt it. Plugin system is the product.
## Database
- **Active database:** `shopdb_flask` (MySQL, asset-based schema)
- **Legacy database:** `shopdb` (Classic ASP schema, used only for one-time data import via `scripts/import_from_mysql.py`)
- **Connection:** `.env` file. See `.env.example`.
Architecture decisions live in `docs/adr/`. Read those before making schema or contract changes.
- ADR-001: Asset model is the platform contract (Machine retires) - ACCEPTED
- ADR-002: Plugin contract versioning (semver) - ACCEPTED
- ADR-003: Plugin distribution model (in-tree bundled + filesystem-based external) - ACCEPTED
- ADR-004: Deployment topology (per-site instances, not multi-tenant) - ACCEPTED
- ADR-005: Equipment vs measuringtools plugin scope - ACCEPTED
- ADR-006: Plugin collector contract pattern - ACCEPTED
- ADR-007: Product versioning and releases - ACCEPTED
- ADR-008: Plugin migration ownership (per-plugin chains) - ACCEPTED
- ADR-009: Frontend plugin route gating - ACCEPTED
- ADR-010: Frontend plugin hook contract - ACCEPTED
- ADR-011: Machines rename + modeltypes retyping - ACCEPTED
- ADR-012: GE-Enforce manifest ownership in shopdb - ACCEPTED
- ADR-013: Plugin catalog, curated shelf, and lean per-site builds - PROPOSED
- ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, prune not-installed plugin tables) - ACCEPTED
- ADR-015: Where a site's own data is allowed to live (setting with a neutral default, site-namespaced dir, or seed) - ACCEPTED
- ADR-016: Credential delivery to the fleet (ShopDB resolves targeting, encrypted at rest, dedicated fetch scope) - PROPOSED
## Coding convention
`CONTRIBUTING.md` defines naming rules (DB tables, columns, Python, JS, Vue, API). Pre-commit hook at `scripts/check-naming-and-style.sh` enforces them. Read `CONTRIBUTING.md` before naming any new identifier.
## Current state (as of 2026-07-13)
Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) largely complete; the last big milestone is the legacy-data import + a production pilot.
### Phases done
- **Phase 0**: 6 ADRs accepted, naming convention v1, pre-commit style hook
- **Phase 1**: 8 smoke tests + 7 production-config tests, Flask-SQLAlchemy 3 fixtures, uv lockfile, hardened ProductionConfig
- **Phase 2**: contract surface defined (`__contract_version__`, `BasePlugin` hooks, `docs/PLUGIN-HOOKS.md`), 51 contract tests
- **Phase 3**: manifest-first loader, fail-loud/isolate policy, contract-version range checking, auto-register core blueprints, `shopdb.api` namespace, `BasePlugin.get_setting/set_setting` helpers
- **Phase 4**: `flask plugin new <name>` CLI, scaffold templates, 14 canary tests, `docs/PLUGIN-QUICKSTART.md`
- **Phase 5**: ADRs moved to `docs/adr/`, Alembic baseline migration, per-site deploy artifacts (`Dockerfile`, `docker-compose.yml`, `docs/DEPLOY.md`)
### Active state
- 1159 tests, naming/style check green, Gitea Actions CI (backend + naming + frontend build + a lean-build job + a migrations-mysql job that runs the real fresh upgrade on utf8mb4 MySQL 8)
- GE-Enforce HTTPS cutover: the displays/kiosks cohort now fetches manifest + inline payloads entirely over HTTPS (share-less); the `gea-shopfloor-display` scope is authored in code (`plugins/geenforce/seed_display_scope.py`) and published via `seed_display_scope(publish=True)`. Other fleet PC types still enforce from the SMB share and only report. See `docs/geenforce-api-cutover.md`.
- `__contract_version__` at 0.16.0 (0.12.0 mailer, 0.13.0 User/Role, 0.14.0 send_webhook, 0.15.0 authorized_service_token, 0.16.0 get_settings_defaults) (product `__version__` 0.7.0, tags v0.5.0/v0.6.0/v0.7.0 - distinct series, ADR-007)
- 13 bundled plugins all satisfy contract: computers, employees, geenforce, knowledgebase, machines, measuringtools, network, notifications, printedparts, printers, slides, usb, warranty
- Core Alembic chain: baseline `68b3947ae14f` -> head `7d26_settings_description_text` (33 core migrations). Each plugin owns its own chain (ADR-008); deploy runs `flask db upgrade` then `flask plugin upgrade-all`. Reproducible + idempotent from empty (env.py relaxes session sql_mode so the chain runs on strict MySQL 8).
- Lean per-site builds (ADR-013 + ADR-014): `scripts/build-site.sh` (backend) + `SITE_PLUGINS` via `scripts/stage-frontend.mjs` (frontend) ship only chosen plugins; `flask plugin prune-schema` drops non-installed plugins' tables at provisioning. Sidebar nav / settings / Displays all gate on staged routes. Manifest-less `plugins/<name>/frontend/` dirs (e.g. `applications`) are core and always ship.
- Windows sites install from a single air-gapped installer `.exe` built per site from its plugin profile (`deploy/windows/installer/`, built by `build-installer.sh` or `build-installer.ps1`). Operator docs: `docs/INSTALL-WINDOWS.md` + `docs/OPERATE-WINDOWS.md` - these are canonical for a NEW site. `docs/INSTALL-WINDOWS-IIS.md` and `docs/DEPLOY-WINDOWS-IIS.md` are the MANUAL procedure, kept for hand-built servers only. The installer verifies its third-party payload against `bundle-lock.json` and installs wheels with `pip --require-hashes`; every build stages a CycloneDX SBOM (`sbom.cdx.json`) onto the server.
- Legacy import: `docs/IMPORT-API.md` is the schema-agnostic import contract; `docs/IMPORT-ADOPTION.md` + `docs/PILOT-DEPLOY.md` cover adopting a site; `scripts/site_imports/wjf/` is the West Jefferson reference loader (all 16 stages, validated end-to-end including on a Windows + MySQL 8 VM).
- API is migration-complete: an admin PAT + docs/IMPORT-API.md let a script import the whole legacy DB (X-Import-Mode preserves timestamps).
- Pre-1.0 framework; sister sites should pin tight `core_version` ranges until contract reaches 1.0
### Deferred
- Equipment data migration (one-shot script for legacy ASP shopdb -> assets). Per ADR-001, only `category='Equipment' AND machinenumber IS NOT NULL` migrates. Skill `migrating-asset-schema` documents the pattern; the actual one-shot script lives in `scripts/migration/` when run.
- Printers retirement: legacy `PrinterData` model + frontend changes. Coordinated with the equipment data migration.
- (DONE 2026-07-11) `measuringtools` plugin (ADR-005) is built and bundled; docs/PLUGIN-GUIDE.md narrates its construction as the plugin tutorial.
- (DONE) Frontend plugin hook contract (ADR-010): get_settings_cards / get_asset_panels / get_map_overlays / get_asset_presentation shipped; generic renderers for panels/overlays land incrementally.
- (DONE) Per-plugin Alembic chains (ADR-008): every bundled plugin carries its own chain; no plugin uses db.create_all().
- Legacy ASP data import against the renamed schema (unblocked; run via docs/IMPORT-API.md) + a production pilot deployment.
## Quick start
```bash
# Start dev environment
~/start-dev-env.sh
# Activate venv and install deps
cd /home/camp/projects/shopdb-flask
source venv/bin/activate
pip install -r requirements.txt
# Configure environment
cp .env.example .env
# Edit .env with DB credentials, JWT secrets
# Create / update database tables via the Alembic chain
flask db upgrade
# Seed RBAC, default settings, and reference data (all idempotent)
flask seed permissions
flask seed settings
flask seed reference-data
# Restart services
pm2 restart shopdb-flask-api shopdb-flask-ui
```
## Service URLs
- Flask API: http://localhost:5001
- Flask UI: http://localhost:5173
- Legacy ASP (data source for one-time import): http://192.168.122.151:8080
## Plugin structure
```
plugins/
<plugin_name>/
__init__.py
plugin.py # BasePlugin implementation
manifest.json # Plugin metadata (name, version, dependencies, api_prefix)
models/
__init__.py # Export all models
<model>.py # SQLAlchemy models
api/
__init__.py
routes.py # Flask Blueprint
services/ # Optional, business logic
schemas/ # Optional, marshmallow schemas
migrations/ # Optional, plugin-specific Alembic migrations
```
Each plugin must have:
- `models/__init__.py` exports all models
- `plugin.py` extends `BasePlugin`
- `manifest.json` with metadata (single source of truth per ADR-002)
- No direct imports from core code (use the contract surface defined in ADR-001)
## Key files
- `shopdb/__init__.py` - app factory, blueprint registration
- `shopdb/plugins/base.py` - BasePlugin ABC, PluginMeta dataclass
- `shopdb/plugins/loader.py` - filesystem discovery, dependency-aware loading
- `shopdb/core/api/assets.py` - example of optional plugin imports
- `frontend/src/router/index.js` - frontend routing
- `frontend/src/components/AppSidebar.vue` - navigation menu
- `docs/adr/` - architecture decision records
## Migration notes
- `migrations/DATA_MIGRATION_GUIDE.md` - one-time import from legacy ASP shopdb
- `migrations/MIGRATE_USB_DEVICES_FROM_EQUIPMENT.md` - USB device migration from equipment table
- `migrations/FIX_LOCATIONONLY_EQUIPMENT_TYPES.md` - LocationOnly equipment type fix
- `migrations/PRODUCTION_MIGRATION_GUIDE.md` - production import methods
- `migrations/rename_underscore_columns.sql` - one-time rename of snake_case columns to lowercase concatenated (per CONTRIBUTING.md)
- `migrations/versions/` - the core Alembic chain (baseline `68b3947ae14f` -> head `7d26_settings_description_text`). Run `flask db upgrade` to apply.
Reference in New Issue View Git Blame Copy Permalink