13e6e039fe
ShopDB is a product several sites adopt, developed at one site against that site's live fleet, so every feature arrives carrying West Jefferson's hostnames, networks and share paths. A scan of plugins/ and shopdb/ returns 19 site literals, of which about 8 are a shipped default or a hardcode rather than a comment: a kiosk URL baked into a PowerShell payload a sister site cannot override, a WJ UNC path as the backup share default, and device.geaerospace.net shipped as a SETTING DEFAULT - inherited silently by anyone who adopts. The mechanisms already existed; the rule did not, so each value landed wherever was convenient. The rule: a setting with a NEUTRAL default first, a site-namespaced directory second, seed data third. Blank must be a working state. Fleet-wide vocabulary such as the gea-shopfloor-* imaging types is not site data and is out of scope - it already has a per-site override. Enforced by an addition to check-naming-and-style.sh, which CI already runs, in REPORT-ONLY mode. Making a 19-item backlog a hard gate the day it lands blocks every unrelated commit until someone clears it; SITE_LITERALS_ENFORCE=1 turns it into a gate once the listed hardcodes are done. The ADR carries that backlog, including that the display FQDN domain is defined three times in three files.
145 lines
9.1 KiB
Markdown
145 lines
9.1 KiB
Markdown
# ShopDB Flask Project
|
|
|
|
Modern rewrite of the classic-ASP shopdb. Built as a framework so sister GE Aerospace sites can adopt it. Plugin system is the product.
|
|
|
|
## Database
|
|
|
|
- **Active database:** `shopdb_flask` (MySQL, asset-based schema)
|
|
- **Legacy database:** `shopdb` (Classic ASP schema, used only for one-time data import via `scripts/import_from_mysql.py`)
|
|
- **Connection:** `.env` file. See `.env.example`.
|
|
|
|
Architecture decisions live in `docs/adr/`. Read those before making schema or contract changes.
|
|
|
|
- ADR-001: Asset model is the platform contract (Machine retires) - ACCEPTED
|
|
- ADR-002: Plugin contract versioning (semver) - ACCEPTED
|
|
- ADR-003: Plugin distribution model (in-tree bundled + filesystem-based external) - ACCEPTED
|
|
- ADR-004: Deployment topology (per-site instances, not multi-tenant) - ACCEPTED
|
|
- ADR-005: Equipment vs measuringtools plugin scope - ACCEPTED
|
|
- ADR-006: Plugin collector contract pattern - ACCEPTED
|
|
- ADR-007: Product versioning and releases - ACCEPTED
|
|
- ADR-008: Plugin migration ownership (per-plugin chains) - ACCEPTED
|
|
- ADR-009: Frontend plugin route gating - ACCEPTED
|
|
- ADR-010: Frontend plugin hook contract - ACCEPTED
|
|
- ADR-011: Machines rename + modeltypes retyping - ACCEPTED
|
|
- ADR-012: GE-Enforce manifest ownership in shopdb - ACCEPTED
|
|
- ADR-013: Plugin catalog, curated shelf, and lean per-site builds - PROPOSED
|
|
- ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, prune not-installed plugin tables) - ACCEPTED
|
|
- ADR-015: Where a site's own data is allowed to live (setting with a neutral default, site-namespaced dir, or seed) - ACCEPTED
|
|
|
|
## Coding convention
|
|
|
|
`CONTRIBUTING.md` defines naming rules (DB tables, columns, Python, JS, Vue, API). Pre-commit hook at `scripts/check-naming-and-style.sh` enforces them. Read `CONTRIBUTING.md` before naming any new identifier.
|
|
|
|
## Current state (as of 2026-07-13)
|
|
|
|
Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) largely complete; the last big milestone is the legacy-data import + a production pilot.
|
|
|
|
### Phases done
|
|
|
|
- **Phase 0**: 6 ADRs accepted, naming convention v1, pre-commit style hook
|
|
- **Phase 1**: 8 smoke tests + 7 production-config tests, Flask-SQLAlchemy 3 fixtures, uv lockfile, hardened ProductionConfig
|
|
- **Phase 2**: contract surface defined (`__contract_version__`, `BasePlugin` hooks, `docs/PLUGIN-HOOKS.md`), 51 contract tests
|
|
- **Phase 3**: manifest-first loader, fail-loud/isolate policy, contract-version range checking, auto-register core blueprints, `shopdb.api` namespace, `BasePlugin.get_setting/set_setting` helpers
|
|
- **Phase 4**: `flask plugin new <name>` CLI, scaffold templates, 14 canary tests, `docs/PLUGIN-QUICKSTART.md`
|
|
- **Phase 5**: ADRs moved to `docs/adr/`, Alembic baseline migration, per-site deploy artifacts (`Dockerfile`, `docker-compose.yml`, `docs/DEPLOY.md`)
|
|
|
|
### Active state
|
|
|
|
- 1159 tests, naming/style check green, Gitea Actions CI (backend + naming + frontend build + a lean-build job + a migrations-mysql job that runs the real fresh upgrade on utf8mb4 MySQL 8)
|
|
- GE-Enforce HTTPS cutover: the displays/kiosks cohort now fetches manifest + inline payloads entirely over HTTPS (share-less); the `gea-shopfloor-display` scope is authored in code (`plugins/geenforce/seed_display_scope.py`) and published via `seed_display_scope(publish=True)`. Other fleet PC types still enforce from the SMB share and only report. See `docs/geenforce-api-cutover.md`.
|
|
- `__contract_version__` at 0.16.0 (0.12.0 mailer, 0.13.0 User/Role, 0.14.0 send_webhook, 0.15.0 authorized_service_token, 0.16.0 get_settings_defaults) (product `__version__` 0.7.0, tags v0.5.0/v0.6.0/v0.7.0 - distinct series, ADR-007)
|
|
- 13 bundled plugins all satisfy contract: computers, employees, geenforce, knowledgebase, machines, measuringtools, network, notifications, printedparts, printers, slides, usb, warranty
|
|
- Core Alembic chain: baseline `68b3947ae14f` -> head `7d26_settings_description_text` (33 core migrations). Each plugin owns its own chain (ADR-008); deploy runs `flask db upgrade` then `flask plugin upgrade-all`. Reproducible + idempotent from empty (env.py relaxes session sql_mode so the chain runs on strict MySQL 8).
|
|
- Lean per-site builds (ADR-013 + ADR-014): `scripts/build-site.sh` (backend) + `SITE_PLUGINS` via `scripts/stage-frontend.mjs` (frontend) ship only chosen plugins; `flask plugin prune-schema` drops non-installed plugins' tables at provisioning. Sidebar nav / settings / Displays all gate on staged routes. Manifest-less `plugins/<name>/frontend/` dirs (e.g. `applications`) are core and always ship.
|
|
- Windows sites install from a single air-gapped installer `.exe` built per site from its plugin profile (`deploy/windows/installer/`, built by `build-installer.sh` or `build-installer.ps1`). Operator docs: `docs/INSTALL-WINDOWS.md` + `docs/OPERATE-WINDOWS.md` - these are canonical for a NEW site. `docs/INSTALL-WINDOWS-IIS.md` and `docs/DEPLOY-WINDOWS-IIS.md` are the MANUAL procedure, kept for hand-built servers only. The installer verifies its third-party payload against `bundle-lock.json` and installs wheels with `pip --require-hashes`; every build stages a CycloneDX SBOM (`sbom.cdx.json`) onto the server.
|
|
- Legacy import: `docs/IMPORT-API.md` is the schema-agnostic import contract; `docs/IMPORT-ADOPTION.md` + `docs/PILOT-DEPLOY.md` cover adopting a site; `scripts/site_imports/wjf/` is the West Jefferson reference loader (all 16 stages, validated end-to-end including on a Windows + MySQL 8 VM).
|
|
- API is migration-complete: an admin PAT + docs/IMPORT-API.md let a script import the whole legacy DB (X-Import-Mode preserves timestamps).
|
|
- Pre-1.0 framework; sister sites should pin tight `core_version` ranges until contract reaches 1.0
|
|
|
|
### Deferred
|
|
|
|
- Equipment data migration (one-shot script for legacy ASP shopdb -> assets). Per ADR-001, only `category='Equipment' AND machinenumber IS NOT NULL` migrates. Skill `migrating-asset-schema` documents the pattern; the actual one-shot script lives in `scripts/migration/` when run.
|
|
- Printers retirement: legacy `PrinterData` model + frontend changes. Coordinated with the equipment data migration.
|
|
- (DONE 2026-07-11) `measuringtools` plugin (ADR-005) is built and bundled; docs/PLUGIN-GUIDE.md narrates its construction as the plugin tutorial.
|
|
- (DONE) Frontend plugin hook contract (ADR-010): get_settings_cards / get_asset_panels / get_map_overlays / get_asset_presentation shipped; generic renderers for panels/overlays land incrementally.
|
|
- (DONE) Per-plugin Alembic chains (ADR-008): every bundled plugin carries its own chain; no plugin uses db.create_all().
|
|
- Legacy ASP data import against the renamed schema (unblocked; run via docs/IMPORT-API.md) + a production pilot deployment.
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
# Start dev environment
|
|
~/start-dev-env.sh
|
|
|
|
# Activate venv and install deps
|
|
cd /home/camp/projects/shopdb-flask
|
|
source venv/bin/activate
|
|
pip install -r requirements.txt
|
|
|
|
# Configure environment
|
|
cp .env.example .env
|
|
# Edit .env with DB credentials, JWT secrets
|
|
|
|
# Create / update database tables via the Alembic chain
|
|
flask db upgrade
|
|
|
|
# Seed RBAC, default settings, and reference data (all idempotent)
|
|
flask seed permissions
|
|
flask seed settings
|
|
flask seed reference-data
|
|
|
|
# Restart services
|
|
pm2 restart shopdb-flask-api shopdb-flask-ui
|
|
```
|
|
|
|
## Service URLs
|
|
|
|
- Flask API: http://localhost:5001
|
|
- Flask UI: http://localhost:5173
|
|
- Legacy ASP (data source for one-time import): http://192.168.122.151:8080
|
|
|
|
## Plugin structure
|
|
|
|
```
|
|
plugins/
|
|
<plugin_name>/
|
|
__init__.py
|
|
plugin.py # BasePlugin implementation
|
|
manifest.json # Plugin metadata (name, version, dependencies, api_prefix)
|
|
models/
|
|
__init__.py # Export all models
|
|
<model>.py # SQLAlchemy models
|
|
api/
|
|
__init__.py
|
|
routes.py # Flask Blueprint
|
|
services/ # Optional, business logic
|
|
schemas/ # Optional, marshmallow schemas
|
|
migrations/ # Optional, plugin-specific Alembic migrations
|
|
```
|
|
|
|
Each plugin must have:
|
|
|
|
- `models/__init__.py` exports all models
|
|
- `plugin.py` extends `BasePlugin`
|
|
- `manifest.json` with metadata (single source of truth per ADR-002)
|
|
- No direct imports from core code (use the contract surface defined in ADR-001)
|
|
|
|
## Key files
|
|
|
|
- `shopdb/__init__.py` - app factory, blueprint registration
|
|
- `shopdb/plugins/base.py` - BasePlugin ABC, PluginMeta dataclass
|
|
- `shopdb/plugins/loader.py` - filesystem discovery, dependency-aware loading
|
|
- `shopdb/core/api/assets.py` - example of optional plugin imports
|
|
- `frontend/src/router/index.js` - frontend routing
|
|
- `frontend/src/components/AppSidebar.vue` - navigation menu
|
|
- `docs/adr/` - architecture decision records
|
|
|
|
## Migration notes
|
|
|
|
- `migrations/DATA_MIGRATION_GUIDE.md` - one-time import from legacy ASP shopdb
|
|
- `migrations/MIGRATE_USB_DEVICES_FROM_EQUIPMENT.md` - USB device migration from equipment table
|
|
- `migrations/FIX_LOCATIONONLY_EQUIPMENT_TYPES.md` - LocationOnly equipment type fix
|
|
- `migrations/PRODUCTION_MIGRATION_GUIDE.md` - production import methods
|
|
- `migrations/rename_underscore_columns.sql` - one-time rename of snake_case columns to lowercase concatenated (per CONTRIBUTING.md)
|
|
- `migrations/versions/` - the core Alembic chain (baseline `68b3947ae14f` -> head `7d26_settings_description_text`). Run `flask db upgrade` to apply.
|