c73b53f613777098fd963b7d39756a28610725da
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
cproudlock
|
417f8a3dd4 |
Keep a site's own files when its container is replaced
`db_data` was a volume and the instance directory was not, so the documented update path - `docker compose build api && up -d api` - recreated the container and discarded everything the site had written. `plugins.json` is only the loud part: maps, branding, model and application images, employee photos, warranty proofs, slides, printed-part files and the Dell OAuth token all live under instance_path too. MySQL rows survive and point at files that are gone, so the second symptom is images 404ing rather than an error anybody sees. Reported by an adopting site, which read it as having updated too fast. It had not; nothing it could have done differently would have kept those files. DEPLOY.md had been telling sites to back up `instance/` since it was written. The template never gave them anything to back up. The air-gap `migrate` service mounts the volume too, because `flask plugin upgrade-all` rewrites plugins.json and that service exits immediately after. The image now creates instance/ ITSELF, owned by the app user. Docker seeds an empty named volume from image content at the mountpoint, ownership included; with no such directory in the image the mountpoint is created root-owned 0755 and the container, which runs as shopdb, cannot write into its own instance directory. Caught by running the built image rather than by reading it: the volume mounted clean and `touch` came back Permission denied. Verified fixed the same way. A stack that predates the volume needs its files moved across ONCE, while the old container still exists - the volume is seeded from image content, and the image ships instance/ empty, so it comes up empty rather than inheriting the old container's writable layer. DEPLOY.md carries the procedure, including the chown after `docker compose cp`, which writes files under the copying user's numeric uid rather than the app user's. Also here, found while checking what an upgrade actually runs: the connected update steps ran `flask db upgrade` and stopped. Per-plugin Alembic chains (ADR-008) are not part of that, so a connected site taking an image with a bumped plugin migration ran the core chain and silently skipped every plugin chain. The air-gap stack had it right all along. Both commands are in Step 9 now, plus a `db current` check against `db heads`. |
||
|
cproudlock
|
ee083ea80e |
docs: stop publishing internal references to a public wiki
docs/ is excluded from the code bundle and its scrub gate, because it goes to the GitHub wiki instead - via a generator that has no gate at all. So the one part of the repository written in prose, by people, about internal infrastructure, was the one part nothing checked. What was reaching a public wiki: the internal git server's URL and hostname, .gitea workflow paths, developer home directories in the GE-Enforce cutover reference, and a dev database root password inside a copy-pasteable command in the import guide. All replaced with neutral equivalents. tests/test_docs_publishable.py is now the gate, at the source, in CI - a wiki page cannot be un-published, so catching this after the fact is not good enough. PROJECT-REVIEW.md also referred to internal tooling by name throughout; those references are generalised. It remains an internal candid assessment of this project that is nonetheless published, which is worth a separate decision. |
||
|
cproudlock
|
e005d1846a |
docs: wiki staleness sweep (Fable-orchestrated Opus audit)
Audited all 40 docs/ against the live codebase; fixed factual staleness in 23, 14 were clean. Highlights (all verified against code): - equipment -> machines (ADR-011 rename) in INSTALL/DEPLOY-WINDOWS-IIS, PLUGIN-GUIDE, GE-ENFORCE, ROADMAP. - Versions refreshed: contract 0.10.0 -> 0.13.0, product 0.5.0 -> 0.7.0, plus plugin example core_version pins. - Bundled set corrected to the current 13 (PLUGINS.md 7 -> 13 rows; DEPLOY eleven -> thirteen). - Per-plugin Alembic chain workflow (ADR-008) replacing stale core-chain steps in PLUGIN-QUICKSTART / BACKUP-RESTORE; deploy adds plugin upgrade-all. - Frontend plugin staging (ADR-010) replacing 'no frontend plugin system yet' in PLUGIN-GUIDE; view/route paths repointed to plugins/<name>/frontend/. - Corrected file paths (MapView.vue, manifest_schema.json), CLI (shelf-list), API gating (GET /api/plugins is optional-jwt), WJF 15 -> 16 stages, and retired Collector/PC-Types settings pages (ADR-012). - ge-enforce proposal marked ACCEPTED/built. |
||
|
cproudlock
|
c386e211df |
ADR-014 Phase 2: flask plugin prune-schema for lean per-site DBs
A lean site still gets every plugin's tables from the shared core Alembic baseline. prune-schema drops the tables of plugins not installed on this site, leaving core + chosen-plugin tables, with no edit to any released migration (the relocate-into-plugin-baselines alternative would mean rewriting ~15 released core migrations for a cosmetic gain - see ADR-014). - shopdb/plugins/cli.py: prune-schema command. Dry-run by default; --yes to execute; refuses non-empty tables without --force. Drops by table name (no plugin import) so it works on a lean image. MySQL: private AUTOCOMMIT engine (db.engine's pooled connections sit idle-in-transaction in a CLI context and would deadlock the DROP on a metadata lock). SQLite: db.engine, restoring the prior foreign_keys pragma so the StaticPool connection is not left changed. - tests/test_plugin_prune_schema.py: drop-only-not-installed, full no-op, refuse-non-empty, force-drops-non-empty. - docs/DEPLOY.md: lean provisioning step after upgrade-all. - ADR-014 ACCEPTED; index updated. Verified on MySQL: full install then prune = no-op (86 tables); lean install (machines+printers) then prune drops the other 19 plugin tables; second run no-op. Full suite 1077 passed. |
||
|
cproudlock
|
48d3160bc5 |
Rename the equipment domain to machines; retype the models catalog (ADR-011)
The equipment plugin is now the machines plugin, ending the UI-vs-code vocabulary split while the contract is pre-1.0 and nothing external depends on the old names. - plugins/equipment -> plugins/machines: manifest, class, /api/machines, machines.* permissions, registry key (with an auto-migrating load shim for existing installs). - Tables: equipment -> machines (equipmentid -> machineid) and equipmenttypes -> machinetypes, renamed in the plugin's own migration chain (machines0002rename), idempotent for both upgrading and fresh installs. - The legacy core machinetypes lookup actually types the vendor MODELS catalog, so it is renamed losslessly to modeltypes (models.modeltypeid, /api/modeltypes, Model Types settings page) rather than collapsed, freeing the machinetypes name. Core migration 7d17_machines_rename also flips data in place: assettypes row equipment -> machine, auditlog entitytype, identifier_/search_ settings keys, permission rows, and renames alembic_version_equipment. - Frontend: machinesApi/modeltypesApi, item.machine response shape, assettype value compares 'equipment' -> 'machine' (map, search, custom fields, relationships), routes machines.js with plugin gating retagged, /print/machine-badge, Machine Types (subtypes) and Model Types (catalog) settings pages, machines-by-type report id. - Docs swept; ADRs left as history per the authoring rule. Upgrade: flask db upgrade then flask plugin upgrade-all. Verified: dev DB flipped live (262 machines, 35 modeltypes, 95 models retyped, zero equipment tables remain); fresh scratch-MySQL install produces the new names; 341 tests green; naming/style green; frontend builds; live E2E on machines list/detail, PC relationships, map, reports, and both settings pages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
cproudlock
|
22e623c1f6 |
Plugin framework maturation, reports overhaul, theming, and USB frontend repair
Framework: - Per-plugin Alembic migration chains (ADR-008): every bundled plugin carries its own chain with a stamp-only anchor at the ownership cutover; new plugin schema lands in plugins/<name>/migrations/, never the core chain. Deploys add flask plugin upgrade-all. Fixed a latent bug in the shared alembic template (engine URL resolution) and taught the metadata filter to include FK-referenced core tables. - Frontend plugin route gating (ADR-009): plugin routes carry meta.plugin; a disabled plugin's pages redirect to the dashboard via a cached, fail-open check against the new public GET /api/plugins/enabled. - get_reports() plugin hook (contract 0.5.0 -> 0.6.0): plugins contribute report cards; warranty and toner cards moved off the hardcoded list. Reports: - Hub grouped by category with search; inline reports render at the top, are URL-backed (?report=id, back-button and deep links work), expose their server-side filter params as controls, and export CSV. Warranty and Toner pages gained CSV export. - Deleted the dead legacy Warranty Status report (always-zero buckets from a retired column). Theming and fonts: - Inter (variable) bundled locally via @fontsource, replacing the Google Fonts Roboto import - air-gapped installs now render correctly; tables use tabular numerals. - Optional brand_primary_dark_color, brand_accent_color, brand_sidebar_color settings applied to CSS vars at bootstrap. USB frontend repair (views were reading a dead legacy shape): - List/detail/form and the employee profile USB panels remapped to the real API shape (device_id/device_desc/checkinoutlog); employee panels now use /usb/checkouts endpoints; external-mode /usb/checkouts/active honors the badge filter; dead client methods pruned. Also: warranties list page no longer requires login (matches app convention); collector doc rewritten with a GE-Enforce integration guide and paste-ready PowerShell reporter; ADR index and CHANGELOG updated. Verified: 323 tests pass, naming/style green, frontend builds, plugin migration dry-run green on scratch MySQL. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
cproudlock
|
b8c22244a1 |
Multi-site distribution readiness: settings-driven site config, security closeout, release engineering, v0.5.0
Make the app distributable to other GE Aerospace sites (one self-hosted
instance per site, ADR-004). GE values remain the shipped defaults; every
site-specific behavior is now a Setting an admin can change in the UI.
Settings-driven site config:
- Branding: site/QR/badge logos, favicon, primary color (upload endpoints
mirror the map-blueprint pattern; new Settings > Branding section).
- ServiceNow: search/incident/change URL templates ({ticket}), ticket
prefixes, enable toggle. Defaults point at the current
geaerospaceqa.service-now.com global search. Disabled = plain-text tickets.
- Employee-id regex (employeeid_pattern), printer hostname template,
QR label targets (qr_target_printer / qr_target_usb, blank = asset page,
else URL template with placeholders), usb_label_style (barcode|qr).
- West Jefferson floor-plan PNGs removed from the tree; generic placeholder
ships as the map default and sites upload their own blueprint.
Security closeout:
- dashboarddefaults writes now require admin.
- Collector: generic error messages (no str(exc) leak); API key accepted
via X-API-Key header only (BREAKING: querystring api_key removed).
- IP-based login rate limiting (AUTH_RATELIMIT_* knobs) atop account lockout.
- Setting.set() creation race fixed (IntegrityError retry).
Release engineering and docs:
- __version__ 0.5.0 (distinct from __contract_version__, ADR-007),
CHANGELOG.md, Gitea Actions CI config, frontend version aligned.
- One wizard-first install story across README/DEPLOY; new CONFIG.md,
UPGRADE.md, BACKUP-RESTORE.md; CLAUDE.md and ROADMAP de-staled.
- Dockerfile multi-stage build now bundles the frontend; compose binds
MySQL to 127.0.0.1; stale database/schema.sql and one-off SQL removed.
Debt and fixes:
- .query.get() -> db.session.get() sweep; datetime.utcnow() removed
(naive-UTC via timezone-aware now); users.py on authz decorators.
- Fixed 4 stale tests (slides feed shape, shopfloor splitperemployee,
plugin contract purity) and the USB label page field mapping (both usb
modes emit the cmmc shape: device_id/device_desc).
- Health endpoint reports the real version.
248 tests pass; naming/style check green; frontend builds; fresh-DB
flask db upgrade + seeds verified; QR targets verified by decoding
rendered codes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
cproudlock
|
78a0ee8d83 |
Add custom fields + warranty plugin, rework settings into two-pane shell
Feature work from the 2026-07 session: Settings IA - Replace the flat 27-card settings hub with a persistent two-pane shell (SettingsLayout.vue): grouped, searchable left rail + content pane. - Nest all settings/* routes under the shell via router post-processing; shared nav catalog in settingsNav.js. Group by asset class (PCs, Printers, Equipment, Network) so per-type settings stop scattering. Custom fields (core) - customfields + customfieldvalues tables (migration 7d14), CRUD API at /api/customfields, per-asset value get/save. - Settings management page + reusable CustomFieldsSection (detail) and CustomFieldsInputs (form) wired into all four asset types. Warranty (new plugin) - plugins/warranty: warranties + warrantyassets (migration 7d15), derived coverage status, provider abstraction (manual now; Dell/Lenovo/HP stubs). - API CRUD + per-asset panel + report buckets; WarrantyPanel on all four detail pages; Warranties management page; Warranty report + Reports card. - Seed warranty.* permissions. Printer drivers - printerdrivers table (migration 7d13) linked to printer models; drivers now surface on the matching printer's detail page. Other - PCDetail rebalanced (Network + Status + Warranty + custom fields on the right). - Rename PCs list "Features" column to "Remote Access"; fix badge hover underline. - Drop equipment islocationonly field. - Centralize asset-type label/route maps into utils/assetTypes.js. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
cproudlock
|
5fa5160420 |
Apply skill-driven review fixes: security, hook isolation, tests, docs
Addresses findings from a 6-lens review against the project skills (defining-asset-contract, enforcing-plugin-contract, hardening-flask-config, integrating-plugin-hooks, pinning-flask-behavior, simplifying-python). Security (hardening-flask-config): - Load per-plugin COLLECTOR_API_KEY_<PLUGIN> from env in create_app. from_object only copies class attributes, so per-plugin keys (ADR-006) were dead in real deploys and silently fell back to the shared key. - EMPLOYEE_DB_USER/PASSWORD no longer default to root/rootpassword (no safe default for a secret; unset fails loud). Documented in .env.example + DEPLOY.md. - COLLECTOR_API_KEY + per-plugin + EMPLOYEE_DB_* added to .env.example/DEPLOY.md. Hook isolation (integrating-plugin-hooks): - collector _collector_plugins and dashboard get_navigation now re-raise in dev/test and log+isolate in prod, instead of silently swallowing a broken plugin hook. Plugin loader (enforcing-plugin-contract): - enable_plugin/install_plugin read dependencies+version from the manifest instead of instantiating the plugin class. - _register_plugin_components rejects a second plugin claiming an already-used api_prefix (reset per app in init_app). Tests (pinning-flask-behavior): - test_identifiers.py: gauge/maintenance round-trip on computer/printer/network create+update; per-type seed yields the 12 identifier keys. - contract tests for apply_collector_payload presence + schema-declarers-implement. - security tests for per-plugin key env loading + no employee-db password default. Docs/contract sync (defining-asset-contract): - PLUGIN-HOOKS.md documents apply_collector_payload; stale 0.2.0 -> 0.3.0. - ADR-006 documents apply_collector_payload + single-dispatch rationale. - ADR-001 enumerates the expanded shopdb.api import surface. Simplify (simplifying-python): - De-duplicate the 21-entry settings defaults: shared build_default_settings() used by both the /settings/seed route and the CLI (were drifting copies). - Remove dead AssetStatus import + redundant AssetType local import in computers plugin; comment the statusid=1 collector default. 153 tests pass (was 145), naming/style green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
06c16bde8a |
Phase 5: Alembic baseline, per-site deploy, ADRs to docs/adr
Migration runner ready and a sister site can deploy from a clean
checkout with one .env file.
ADRs relocated (migrations/adr/ -> docs/adr/):
- migrations/ is now Alembic territory, not docs.
- All cross-references updated: CLAUDE.md, docs/PLUGIN-HOOKS.md,
docs/PLUGIN-QUICKSTART.md.
Alembic initialized (migrations/):
- env.py, script.py.mako, alembic.ini copied from Flask-Migrate
templates so `flask db migrate` and `flask db upgrade` work without
a one-time `flask db init` (which would clash with the existing
migrations/ directory).
- Baseline migration generated via autogenerate, captures all 47
tables (core models + 6 plugins) as the upgrade target. Ready for
per-site `flask db upgrade` from an empty schema.
Deploy artifacts:
- Dockerfile: python:3.12-slim base, gunicorn server, non-root user,
healthcheck against /api/auth/login. Single image bundles all six
plugins; sites enable via `flask plugin install <name>`.
- docker-compose.yml: MySQL 8 + API container, healthcheck-gated
startup, env-driven secrets that fail loud on missing values
(`${SECRET_KEY:?}` form).
- .env.example: full env-var inventory with comments. Calls out
required vs optional. Matches what ProductionConfig.validate
enforces.
docs/DEPLOY.md:
- Step-by-step per-site runbook: clone, configure .env, bring up
stack, run migrations, seed reference data, install plugins,
create admin, front with TLS, backups, updates.
- Common-issues table.
- Cross-links to ADR-004 (per-site rationale), ADR-003 (plugin
distribution), and the config source.
Skills:
- migrating-asset-schema: Alembic + one-shot data migration policy.
Rules: additive first, renames are three steps, destructive ops
need rollback, equipment migration filter per ADR-001 + ADR-005.
- hardening-flask-config: production validation, CORS allowlist
policy, JWT cookie hardening, per-site deploy isolation per ADR-004.
CLAUDE.md updated to reflect the post-Phase-5 state. No tests added
this commit; the Alembic baseline is exercised by the existing
db.create_all-based test suite (tests do not touch the migration
runner; that's by design until per-plugin migrations land).
Test count unchanged: 101 passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|