aea2905de0
Nine fixes from a review of the installer against its actual audience: DT leads at sister sites who are not Windows, IIS or Python specialists and who will lean on an AI assistant to get through it. TRUTHFULNESS. The preflight was advisory - an operator read 'IIS is not installed', pressed Next, answered five more pages and the install died partway through with Python already on the box. The results page now blocks while anything is failing, repaints on every run instead of latching after the first, and offers 'Check again' so a fixed problem does not mean starting over. On failure the wizard said 'Nothing was left running', which is false in every path because the stages run with -OnFailure never: it now says the server is part-configured, that re-running is safe, and how to remove it. The final page no longer reads 'ShopDB-Flask is ready' after a failed install. SECRETS. The generated MySQL root password went to Write-Host in a process the wizard runs hidden - so nobody saw it - and stdout is forwarded into the setup log operators are told to send to support, so it was permanently recorded for everyone who did not need it. It now goes to an ACL'd file. Database dumps, which contain every user password hash, landed in a ProgramData directory readable by every user on the box; the directory is now locked at creation. UPGRADES ON REMOTE-DATABASE SITES. mysqldump was looked for only under local MySQL install paths, so a site whose database is on another host silently skipped every pre-upgrade backup - after stage 2 had already stopped the pool and replaced the tree. Find-MysqlTool now prefers a client shipped in the bundle, stage 2 stages it onto the server, preflight reports when it is missing, and mysqlclient\ is an optional locked payload. UNINSTALL. A subpath install is an IIS Application, not a site; removing only the site left the application pointing at a deleted directory, so the parent site - at West Jefferson, the live classic ASP - served 503 on that path forever while Add/Remove Programs reported success. Uninstall now reads MOUNT_PATH and removes the application. The firewall rule was created as "$SiteName $SitePort" and removed as the literal 'ShopDB-Flask 8090', which matches nothing. DAY-2 TOOLING. Every shortcut now passes -AppRoot and -SitePort, and the console forwards them through its own elevation and 32-bit relaunches instead of discarding them - a non-default directory or port made it report a healthy site as broken, from a shortcut the installer wrote. 'Open ShopDB-Flask' resolved to a hardcoded localhost:8090 that was wrong for every subpath install; it now asks the console, which reads the address the installer recorded, and no longer demands administrator to open a browser. SMOKE TEST. The parent-site port lookup filtered for an http binding and defaulted to 80, so an https-only parent site failed a working install with a red dialog. DOCS AND /api/docs. The installer was invisible: nothing in docs/, README.md or CLAUDE.md mentioned it, so a DT lead or their assistant landed on the manual IIS runbook and hand-built the very server the installer then refuses to upgrade. docs/INSTALL-WINDOWS.md and docs/OPERATE-WINDOWS.md are now the canonical route, the two manual runbooks are bannered as reference-only, README and CLAUDE.md route by target, and llms.txt tells an assistant which document to follow and to ask for 'check -Json' before diagnosing. Both ship on the server, along with openapi.json and llms.txt - without those the self-hosted /api/docs was broken on every installed box, which matters most to the sites least able to debug it. Stage 5 now checks it actually serves. shopdb-admin.ps1 gains 'check -Json': one structured, secret-free block covering version, publishing method, IIS state, HTTP reachability, database, Python version, plugins and errors. That is the cheapest useful answer to 'the operator will ask an LLM' - it works with no infrastructure, which a install-time MCP server could not.
144 lines
9.0 KiB
Markdown
144 lines
9.0 KiB
Markdown
# ShopDB Flask Project
|
|
|
|
Modern rewrite of the classic-ASP shopdb. Built as a framework so sister GE Aerospace sites can adopt it. Plugin system is the product.
|
|
|
|
## Database
|
|
|
|
- **Active database:** `shopdb_flask` (MySQL, asset-based schema)
|
|
- **Legacy database:** `shopdb` (Classic ASP schema, used only for one-time data import via `scripts/import_from_mysql.py`)
|
|
- **Connection:** `.env` file. See `.env.example`.
|
|
|
|
Architecture decisions live in `docs/adr/`. Read those before making schema or contract changes.
|
|
|
|
- ADR-001: Asset model is the platform contract (Machine retires) - ACCEPTED
|
|
- ADR-002: Plugin contract versioning (semver) - ACCEPTED
|
|
- ADR-003: Plugin distribution model (in-tree bundled + filesystem-based external) - ACCEPTED
|
|
- ADR-004: Deployment topology (per-site instances, not multi-tenant) - ACCEPTED
|
|
- ADR-005: Equipment vs measuringtools plugin scope - ACCEPTED
|
|
- ADR-006: Plugin collector contract pattern - ACCEPTED
|
|
- ADR-007: Product versioning and releases - ACCEPTED
|
|
- ADR-008: Plugin migration ownership (per-plugin chains) - ACCEPTED
|
|
- ADR-009: Frontend plugin route gating - ACCEPTED
|
|
- ADR-010: Frontend plugin hook contract - ACCEPTED
|
|
- ADR-011: Machines rename + modeltypes retyping - ACCEPTED
|
|
- ADR-012: GE-Enforce manifest ownership in shopdb - ACCEPTED
|
|
- ADR-013: Plugin catalog, curated shelf, and lean per-site builds - PROPOSED
|
|
- ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, prune not-installed plugin tables) - ACCEPTED
|
|
|
|
## Coding convention
|
|
|
|
`CONTRIBUTING.md` defines naming rules (DB tables, columns, Python, JS, Vue, API). Pre-commit hook at `scripts/check-naming-and-style.sh` enforces them. Read `CONTRIBUTING.md` before naming any new identifier.
|
|
|
|
## Current state (as of 2026-07-13)
|
|
|
|
Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) largely complete; the last big milestone is the legacy-data import + a production pilot.
|
|
|
|
### Phases done
|
|
|
|
- **Phase 0**: 6 ADRs accepted, naming convention v1, pre-commit style hook
|
|
- **Phase 1**: 8 smoke tests + 7 production-config tests, Flask-SQLAlchemy 3 fixtures, uv lockfile, hardened ProductionConfig
|
|
- **Phase 2**: contract surface defined (`__contract_version__`, `BasePlugin` hooks, `docs/PLUGIN-HOOKS.md`), 51 contract tests
|
|
- **Phase 3**: manifest-first loader, fail-loud/isolate policy, contract-version range checking, auto-register core blueprints, `shopdb.api` namespace, `BasePlugin.get_setting/set_setting` helpers
|
|
- **Phase 4**: `flask plugin new <name>` CLI, scaffold templates, 14 canary tests, `docs/PLUGIN-QUICKSTART.md`
|
|
- **Phase 5**: ADRs moved to `docs/adr/`, Alembic baseline migration, per-site deploy artifacts (`Dockerfile`, `docker-compose.yml`, `docs/DEPLOY.md`)
|
|
|
|
### Active state
|
|
|
|
- 1159 tests, naming/style check green, Gitea Actions CI (backend + naming + frontend build + a lean-build job + a migrations-mysql job that runs the real fresh upgrade on utf8mb4 MySQL 8)
|
|
- GE-Enforce HTTPS cutover: the displays/kiosks cohort now fetches manifest + inline payloads entirely over HTTPS (share-less); the `gea-shopfloor-display` scope is authored in code (`plugins/geenforce/seed_display_scope.py`) and published via `seed_display_scope(publish=True)`. Other fleet PC types still enforce from the SMB share and only report. See `docs/geenforce-api-cutover.md`.
|
|
- `__contract_version__` at 0.15.0 (0.12.0 mailer, 0.13.0 User/Role, 0.14.0 send_webhook, 0.15.0 authorized_service_token) (product `__version__` 0.7.0, tags v0.5.0/v0.6.0/v0.7.0 - distinct series, ADR-007)
|
|
- 13 bundled plugins all satisfy contract: computers, employees, geenforce, knowledgebase, machines, measuringtools, network, notifications, printedparts, printers, slides, usb, warranty
|
|
- Core Alembic chain: baseline `68b3947ae14f` -> head `7d26_settings_description_text` (33 core migrations). Each plugin owns its own chain (ADR-008); deploy runs `flask db upgrade` then `flask plugin upgrade-all`. Reproducible + idempotent from empty (env.py relaxes session sql_mode so the chain runs on strict MySQL 8).
|
|
- Lean per-site builds (ADR-013 + ADR-014): `scripts/build-site.sh` (backend) + `SITE_PLUGINS` via `scripts/stage-frontend.mjs` (frontend) ship only chosen plugins; `flask plugin prune-schema` drops non-installed plugins' tables at provisioning. Sidebar nav / settings / Displays all gate on staged routes. Manifest-less `plugins/<name>/frontend/` dirs (e.g. `applications`) are core and always ship.
|
|
- Windows sites install from a single air-gapped installer `.exe` built per site from its plugin profile (`deploy/windows/installer/`, built by `build-installer.sh` or `build-installer.ps1`). Operator docs: `docs/INSTALL-WINDOWS.md` + `docs/OPERATE-WINDOWS.md` - these are canonical for a NEW site. `docs/INSTALL-WINDOWS-IIS.md` and `docs/DEPLOY-WINDOWS-IIS.md` are the MANUAL procedure, kept for hand-built servers only. The installer verifies its third-party payload against `bundle-lock.json` and installs wheels with `pip --require-hashes`; every build stages a CycloneDX SBOM (`sbom.cdx.json`) onto the server.
|
|
- Legacy import: `docs/IMPORT-API.md` is the schema-agnostic import contract; `docs/IMPORT-ADOPTION.md` + `docs/PILOT-DEPLOY.md` cover adopting a site; `scripts/site_imports/wjf/` is the West Jefferson reference loader (all 16 stages, validated end-to-end including on a Windows + MySQL 8 VM).
|
|
- API is migration-complete: an admin PAT + docs/IMPORT-API.md let a script import the whole legacy DB (X-Import-Mode preserves timestamps).
|
|
- Pre-1.0 framework; sister sites should pin tight `core_version` ranges until contract reaches 1.0
|
|
|
|
### Deferred
|
|
|
|
- Equipment data migration (one-shot script for legacy ASP shopdb -> assets). Per ADR-001, only `category='Equipment' AND machinenumber IS NOT NULL` migrates. Skill `migrating-asset-schema` documents the pattern; the actual one-shot script lives in `scripts/migration/` when run.
|
|
- Printers retirement: legacy `PrinterData` model + frontend changes. Coordinated with the equipment data migration.
|
|
- (DONE 2026-07-11) `measuringtools` plugin (ADR-005) is built and bundled; docs/PLUGIN-GUIDE.md narrates its construction as the plugin tutorial.
|
|
- (DONE) Frontend plugin hook contract (ADR-010): get_settings_cards / get_asset_panels / get_map_overlays / get_asset_presentation shipped; generic renderers for panels/overlays land incrementally.
|
|
- (DONE) Per-plugin Alembic chains (ADR-008): every bundled plugin carries its own chain; no plugin uses db.create_all().
|
|
- Legacy ASP data import against the renamed schema (unblocked; run via docs/IMPORT-API.md) + a production pilot deployment.
|
|
|
|
## Quick start
|
|
|
|
```bash
|
|
# Start dev environment
|
|
~/start-dev-env.sh
|
|
|
|
# Activate venv and install deps
|
|
cd /home/camp/projects/shopdb-flask
|
|
source venv/bin/activate
|
|
pip install -r requirements.txt
|
|
|
|
# Configure environment
|
|
cp .env.example .env
|
|
# Edit .env with DB credentials, JWT secrets
|
|
|
|
# Create / update database tables via the Alembic chain
|
|
flask db upgrade
|
|
|
|
# Seed RBAC, default settings, and reference data (all idempotent)
|
|
flask seed permissions
|
|
flask seed settings
|
|
flask seed reference-data
|
|
|
|
# Restart services
|
|
pm2 restart shopdb-flask-api shopdb-flask-ui
|
|
```
|
|
|
|
## Service URLs
|
|
|
|
- Flask API: http://localhost:5001
|
|
- Flask UI: http://localhost:5173
|
|
- Legacy ASP (data source for one-time import): http://192.168.122.151:8080
|
|
|
|
## Plugin structure
|
|
|
|
```
|
|
plugins/
|
|
<plugin_name>/
|
|
__init__.py
|
|
plugin.py # BasePlugin implementation
|
|
manifest.json # Plugin metadata (name, version, dependencies, api_prefix)
|
|
models/
|
|
__init__.py # Export all models
|
|
<model>.py # SQLAlchemy models
|
|
api/
|
|
__init__.py
|
|
routes.py # Flask Blueprint
|
|
services/ # Optional, business logic
|
|
schemas/ # Optional, marshmallow schemas
|
|
migrations/ # Optional, plugin-specific Alembic migrations
|
|
```
|
|
|
|
Each plugin must have:
|
|
|
|
- `models/__init__.py` exports all models
|
|
- `plugin.py` extends `BasePlugin`
|
|
- `manifest.json` with metadata (single source of truth per ADR-002)
|
|
- No direct imports from core code (use the contract surface defined in ADR-001)
|
|
|
|
## Key files
|
|
|
|
- `shopdb/__init__.py` - app factory, blueprint registration
|
|
- `shopdb/plugins/base.py` - BasePlugin ABC, PluginMeta dataclass
|
|
- `shopdb/plugins/loader.py` - filesystem discovery, dependency-aware loading
|
|
- `shopdb/core/api/assets.py` - example of optional plugin imports
|
|
- `frontend/src/router/index.js` - frontend routing
|
|
- `frontend/src/components/AppSidebar.vue` - navigation menu
|
|
- `docs/adr/` - architecture decision records
|
|
|
|
## Migration notes
|
|
|
|
- `migrations/DATA_MIGRATION_GUIDE.md` - one-time import from legacy ASP shopdb
|
|
- `migrations/MIGRATE_USB_DEVICES_FROM_EQUIPMENT.md` - USB device migration from equipment table
|
|
- `migrations/FIX_LOCATIONONLY_EQUIPMENT_TYPES.md` - LocationOnly equipment type fix
|
|
- `migrations/PRODUCTION_MIGRATION_GUIDE.md` - production import methods
|
|
- `migrations/rename_underscore_columns.sql` - one-time rename of snake_case columns to lowercase concatenated (per CONTRIBUTING.md)
|
|
- `migrations/versions/` - the core Alembic chain (baseline `68b3947ae14f` -> head `7d26_settings_description_text`). Run `flask db upgrade` to apply.
|