ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
10ee3a3c58641f0469641a667ea019989eb2213a
shopdb-flask/CLAUDE.md
cproudlock aea2905de0
Some checks failed
CI / backend (push) Failing after 7s
Details
CI / naming (push) Successful in 2s
Details
CI / frontend (push) Successful in 9s
Details
CI / migrations-mysql (push) Failing after 7s
Details
fix(installer): stop it lying, stop it leaking, and make it findable 
Nine fixes from a review of the installer against its actual audience: DT leads
at sister sites who are not Windows, IIS or Python specialists and who will lean
on an AI assistant to get through it.

TRUTHFULNESS. The preflight was advisory - an operator read 'IIS is not
installed', pressed Next, answered five more pages and the install died partway
through with Python already on the box. The results page now blocks while
anything is failing, repaints on every run instead of latching after the first,
and offers 'Check again' so a fixed problem does not mean starting over. On
failure the wizard said 'Nothing was left running', which is false in every path
because the stages run with -OnFailure never: it now says the server is
part-configured, that re-running is safe, and how to remove it. The final page no
longer reads 'ShopDB-Flask is ready' after a failed install.

SECRETS. The generated MySQL root password went to Write-Host in a process the
wizard runs hidden - so nobody saw it - and stdout is forwarded into the setup
log operators are told to send to support, so it was permanently recorded for
everyone who did not need it. It now goes to an ACL'd file. Database dumps, which
contain every user password hash, landed in a ProgramData directory readable by
every user on the box; the directory is now locked at creation.

UPGRADES ON REMOTE-DATABASE SITES. mysqldump was looked for only under local
MySQL install paths, so a site whose database is on another host silently skipped
every pre-upgrade backup - after stage 2 had already stopped the pool and
replaced the tree. Find-MysqlTool now prefers a client shipped in the bundle,
stage 2 stages it onto the server, preflight reports when it is missing, and
mysqlclient\ is an optional locked payload.

UNINSTALL. A subpath install is an IIS Application, not a site; removing only the
site left the application pointing at a deleted directory, so the parent site -
at West Jefferson, the live classic ASP - served 503 on that path forever while
Add/Remove Programs reported success. Uninstall now reads MOUNT_PATH and removes
the application. The firewall rule was created as "$SiteName $SitePort" and
removed as the literal 'ShopDB-Flask 8090', which matches nothing.

DAY-2 TOOLING. Every shortcut now passes -AppRoot and -SitePort, and the console
forwards them through its own elevation and 32-bit relaunches instead of
discarding them - a non-default directory or port made it report a healthy site
as broken, from a shortcut the installer wrote. 'Open ShopDB-Flask' resolved to a
hardcoded localhost:8090 that was wrong for every subpath install; it now asks
the console, which reads the address the installer recorded, and no longer
demands administrator to open a browser.

SMOKE TEST. The parent-site port lookup filtered for an http binding and
defaulted to 80, so an https-only parent site failed a working install with a red
dialog.

DOCS AND /api/docs. The installer was invisible: nothing in docs/, README.md or
CLAUDE.md mentioned it, so a DT lead or their assistant landed on the manual IIS
runbook and hand-built the very server the installer then refuses to upgrade.
docs/INSTALL-WINDOWS.md and docs/OPERATE-WINDOWS.md are now the canonical route,
the two manual runbooks are bannered as reference-only, README and CLAUDE.md
route by target, and llms.txt tells an assistant which document to follow and to
ask for 'check -Json' before diagnosing. Both ship on the server, along with
openapi.json and llms.txt - without those the self-hosted /api/docs was broken on
every installed box, which matters most to the sites least able to debug it.
Stage 5 now checks it actually serves.

shopdb-admin.ps1 gains 'check -Json': one structured, secret-free block covering
version, publishing method, IIS state, HTTP reachability, database, Python
version, plugins and errors. That is the cheapest useful answer to 'the operator
will ask an LLM' - it works with no infrastructure, which a install-time MCP
server could not.
2026-08-03 14:39:38 -04:00

9.0 KiB
Raw Blame History

ShopDB Flask Project

Modern rewrite of the classic-ASP shopdb. Built as a framework so sister GE Aerospace sites can adopt it. Plugin system is the product.

Database

  • Active database: shopdb_flask (MySQL, asset-based schema)
  • Legacy database: shopdb (Classic ASP schema, used only for one-time data import via scripts/import_from_mysql.py)
  • Connection: .env file. See .env.example.

Architecture decisions live in docs/adr/. Read those before making schema or contract changes.

  • ADR-001: Asset model is the platform contract (Machine retires) - ACCEPTED
  • ADR-002: Plugin contract versioning (semver) - ACCEPTED
  • ADR-003: Plugin distribution model (in-tree bundled + filesystem-based external) - ACCEPTED
  • ADR-004: Deployment topology (per-site instances, not multi-tenant) - ACCEPTED
  • ADR-005: Equipment vs measuringtools plugin scope - ACCEPTED
  • ADR-006: Plugin collector contract pattern - ACCEPTED
  • ADR-007: Product versioning and releases - ACCEPTED
  • ADR-008: Plugin migration ownership (per-plugin chains) - ACCEPTED
  • ADR-009: Frontend plugin route gating - ACCEPTED
  • ADR-010: Frontend plugin hook contract - ACCEPTED
  • ADR-011: Machines rename + modeltypes retyping - ACCEPTED
  • ADR-012: GE-Enforce manifest ownership in shopdb - ACCEPTED
  • ADR-013: Plugin catalog, curated shelf, and lean per-site builds - PROPOSED
  • ADR-014: Schema-lean per-site builds (retire cross-plugin FKs, prune not-installed plugin tables) - ACCEPTED

Coding convention

CONTRIBUTING.md defines naming rules (DB tables, columns, Python, JS, Vue, API). Pre-commit hook at scripts/check-naming-and-style.sh enforces them. Read CONTRIBUTING.md before naming any new identifier.

Current state (as of 2026-07-13)

Refactor phases 0-5 landed; phase 6 (multi-site distribution readiness) largely complete; the last big milestone is the legacy-data import + a production pilot.

Phases done

  • Phase 0: 6 ADRs accepted, naming convention v1, pre-commit style hook
  • Phase 1: 8 smoke tests + 7 production-config tests, Flask-SQLAlchemy 3 fixtures, uv lockfile, hardened ProductionConfig
  • Phase 2: contract surface defined (__contract_version__, BasePlugin hooks, docs/PLUGIN-HOOKS.md), 51 contract tests
  • Phase 3: manifest-first loader, fail-loud/isolate policy, contract-version range checking, auto-register core blueprints, shopdb.api namespace, BasePlugin.get_setting/set_setting helpers
  • Phase 4: flask plugin new <name> CLI, scaffold templates, 14 canary tests, docs/PLUGIN-QUICKSTART.md
  • Phase 5: ADRs moved to docs/adr/, Alembic baseline migration, per-site deploy artifacts (Dockerfile, docker-compose.yml, docs/DEPLOY.md)

Active state

  • 1159 tests, naming/style check green, Gitea Actions CI (backend + naming + frontend build + a lean-build job + a migrations-mysql job that runs the real fresh upgrade on utf8mb4 MySQL 8)
  • GE-Enforce HTTPS cutover: the displays/kiosks cohort now fetches manifest + inline payloads entirely over HTTPS (share-less); the gea-shopfloor-display scope is authored in code (plugins/geenforce/seed_display_scope.py) and published via seed_display_scope(publish=True). Other fleet PC types still enforce from the SMB share and only report. See docs/geenforce-api-cutover.md.
  • __contract_version__ at 0.15.0 (0.12.0 mailer, 0.13.0 User/Role, 0.14.0 send_webhook, 0.15.0 authorized_service_token) (product __version__ 0.7.0, tags v0.5.0/v0.6.0/v0.7.0 - distinct series, ADR-007)
  • 13 bundled plugins all satisfy contract: computers, employees, geenforce, knowledgebase, machines, measuringtools, network, notifications, printedparts, printers, slides, usb, warranty
  • Core Alembic chain: baseline 68b3947ae14f -> head 7d26_settings_description_text (33 core migrations). Each plugin owns its own chain (ADR-008); deploy runs flask db upgrade then flask plugin upgrade-all. Reproducible + idempotent from empty (env.py relaxes session sql_mode so the chain runs on strict MySQL 8).
  • Lean per-site builds (ADR-013 + ADR-014): scripts/build-site.sh (backend) + SITE_PLUGINS via scripts/stage-frontend.mjs (frontend) ship only chosen plugins; flask plugin prune-schema drops non-installed plugins' tables at provisioning. Sidebar nav / settings / Displays all gate on staged routes. Manifest-less plugins/<name>/frontend/ dirs (e.g. applications) are core and always ship.
  • Windows sites install from a single air-gapped installer .exe built per site from its plugin profile (deploy/windows/installer/, built by build-installer.sh or build-installer.ps1). Operator docs: docs/INSTALL-WINDOWS.md + docs/OPERATE-WINDOWS.md - these are canonical for a NEW site. docs/INSTALL-WINDOWS-IIS.md and docs/DEPLOY-WINDOWS-IIS.md are the MANUAL procedure, kept for hand-built servers only. The installer verifies its third-party payload against bundle-lock.json and installs wheels with pip --require-hashes; every build stages a CycloneDX SBOM (sbom.cdx.json) onto the server.
  • Legacy import: docs/IMPORT-API.md is the schema-agnostic import contract; docs/IMPORT-ADOPTION.md + docs/PILOT-DEPLOY.md cover adopting a site; scripts/site_imports/wjf/ is the West Jefferson reference loader (all 16 stages, validated end-to-end including on a Windows + MySQL 8 VM).
  • API is migration-complete: an admin PAT + docs/IMPORT-API.md let a script import the whole legacy DB (X-Import-Mode preserves timestamps).
  • Pre-1.0 framework; sister sites should pin tight core_version ranges until contract reaches 1.0

Deferred

  • Equipment data migration (one-shot script for legacy ASP shopdb -> assets). Per ADR-001, only category='Equipment' AND machinenumber IS NOT NULL migrates. Skill migrating-asset-schema documents the pattern; the actual one-shot script lives in scripts/migration/ when run.
  • Printers retirement: legacy PrinterData model + frontend changes. Coordinated with the equipment data migration.
  • (DONE 2026-07-11) measuringtools plugin (ADR-005) is built and bundled; docs/PLUGIN-GUIDE.md narrates its construction as the plugin tutorial.
  • (DONE) Frontend plugin hook contract (ADR-010): get_settings_cards / get_asset_panels / get_map_overlays / get_asset_presentation shipped; generic renderers for panels/overlays land incrementally.
  • (DONE) Per-plugin Alembic chains (ADR-008): every bundled plugin carries its own chain; no plugin uses db.create_all().
  • Legacy ASP data import against the renamed schema (unblocked; run via docs/IMPORT-API.md) + a production pilot deployment.

Quick start

# Start dev environment
~/start-dev-env.sh

# Activate venv and install deps
cd /home/camp/projects/shopdb-flask
source venv/bin/activate
pip install -r requirements.txt

# Configure environment
cp .env.example .env
# Edit .env with DB credentials, JWT secrets

# Create / update database tables via the Alembic chain
flask db upgrade

# Seed RBAC, default settings, and reference data (all idempotent)
flask seed permissions
flask seed settings
flask seed reference-data

# Restart services
pm2 restart shopdb-flask-api shopdb-flask-ui

Service URLs

Plugin structure

plugins/
  <plugin_name>/
    __init__.py
    plugin.py          # BasePlugin implementation
    manifest.json      # Plugin metadata (name, version, dependencies, api_prefix)
    models/
      __init__.py      # Export all models
      <model>.py       # SQLAlchemy models
    api/
      __init__.py
      routes.py        # Flask Blueprint
    services/          # Optional, business logic
    schemas/           # Optional, marshmallow schemas
    migrations/        # Optional, plugin-specific Alembic migrations

Each plugin must have:

  • models/__init__.py exports all models
  • plugin.py extends BasePlugin
  • manifest.json with metadata (single source of truth per ADR-002)
  • No direct imports from core code (use the contract surface defined in ADR-001)

Key files

  • shopdb/__init__.py - app factory, blueprint registration
  • shopdb/plugins/base.py - BasePlugin ABC, PluginMeta dataclass
  • shopdb/plugins/loader.py - filesystem discovery, dependency-aware loading
  • shopdb/core/api/assets.py - example of optional plugin imports
  • frontend/src/router/index.js - frontend routing
  • frontend/src/components/AppSidebar.vue - navigation menu
  • docs/adr/ - architecture decision records

Migration notes

  • migrations/DATA_MIGRATION_GUIDE.md - one-time import from legacy ASP shopdb
  • migrations/MIGRATE_USB_DEVICES_FROM_EQUIPMENT.md - USB device migration from equipment table
  • migrations/FIX_LOCATIONONLY_EQUIPMENT_TYPES.md - LocationOnly equipment type fix
  • migrations/PRODUCTION_MIGRATION_GUIDE.md - production import methods
  • migrations/rename_underscore_columns.sql - one-time rename of snake_case columns to lowercase concatenated (per CONTRIBUTING.md)
  • migrations/versions/ - the core Alembic chain (baseline 68b3947ae14f -> head 7d26_settings_description_text). Run flask db upgrade to apply.
Reference in New Issue View Git Blame Copy Permalink