259fe0de8f363ea2f407503d6f7800c18f3974ba
8 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
cproudlock
|
784964c907 |
Bring the wilm branch into main: endpoints plugin, agent, policies, CSV import
Ports the corporate-GitHub wilm branch onto main, keeping the two branches
separate. Thirteen upstream commits arrive with their authorship intact; the
review fixes and the portability work below are on top.
What arrives:
endpoints plugin - a ShopDB agent reporting endpoint state, telemetry,
services and interfaces, with a command channel the agent
long-polls. Ingest stays on the generic collector route
(ADR-006); the agent key is a UI-managed single-scope
token whose revocation is an instant kill switch.
computers policies - Configuration and Business Policies (RMM-style), scope
trees validated at write time, actions checked against
the endpoints command allowlist. Feature-gated: every
route 404s while the setting is off, so enabling it can
never half-expose the surface.
asset CSV import - templates, a collector script carrying the site path, and
a validate-then-commit import. A plugin can own a virtual
column plus a post-write row hook, which is how a PC's
machine number becomes a controls link without a
hand-built relationships sheet.
agent/ - the Windows agent and its installer.
Contract 0.21.0 -> 0.24.0, additive throughout: register_import_hook (0.22),
mint/list/revoke_service_token (0.23), collector_authorized (0.24). The stability
table had no rows for any of them and now does.
MySQL 5.6 portability. The branch stored three policy columns as sa.JSON, the
only JSON columns in the product, and 5.6 has no JSON type: the migration failed
with a bare syntax error near the word JSON. They are TEXT now, through a
JSONText type decorator, so all seventeen call sites still read and write dicts.
The convention elsewhere is already TEXT (guides.blocksjson,
geenforce.manifestjson).
Review fixes carried on top:
- six policy GETs were jwt_required(optional=True), which authenticates a
caller without checking anything; they require computers.view now, and six
redundant jwt_required decorators are gone
- the agent installer took a default BaseUrl and CollectorKey, so a mistyped
run silently pointed an endpoint at the wrong instance; both are mandatory
and a non-https BaseUrl throws unless -AllowInsecureHttp is passed
- a committing CSV import wrote no audit row
Documentation: the 34 new routes are described in docs/api-inventory.json, which
is hand-maintained on purpose, and the generated spec now covers 331 paths and
477 operations across 22 surfaces. DEVELOPMENT-SETUP's 1071 troubleshooting row
said "use MySQL 8 for dev" and now gives the actual fix, including the trap that
5.6 downgrades ROW_FORMAT=DYNAMIC to Compact with a warning instead of an error.
Full suite 2123 passed, vitest 260 passed, all seventeen plugin chains migrate
clean on MySQL 5.6 and the naming, docs, parity and read-surface gates are green.
|
||
|
cproudlock
|
417f8a3dd4 |
Keep a site's own files when its container is replaced
`db_data` was a volume and the instance directory was not, so the documented update path - `docker compose build api && up -d api` - recreated the container and discarded everything the site had written. `plugins.json` is only the loud part: maps, branding, model and application images, employee photos, warranty proofs, slides, printed-part files and the Dell OAuth token all live under instance_path too. MySQL rows survive and point at files that are gone, so the second symptom is images 404ing rather than an error anybody sees. Reported by an adopting site, which read it as having updated too fast. It had not; nothing it could have done differently would have kept those files. DEPLOY.md had been telling sites to back up `instance/` since it was written. The template never gave them anything to back up. The air-gap `migrate` service mounts the volume too, because `flask plugin upgrade-all` rewrites plugins.json and that service exits immediately after. The image now creates instance/ ITSELF, owned by the app user. Docker seeds an empty named volume from image content at the mountpoint, ownership included; with no such directory in the image the mountpoint is created root-owned 0755 and the container, which runs as shopdb, cannot write into its own instance directory. Caught by running the built image rather than by reading it: the volume mounted clean and `touch` came back Permission denied. Verified fixed the same way. A stack that predates the volume needs its files moved across ONCE, while the old container still exists - the volume is seeded from image content, and the image ships instance/ empty, so it comes up empty rather than inheriting the old container's writable layer. DEPLOY.md carries the procedure, including the chown after `docker compose cp`, which writes files under the copying user's numeric uid rather than the app user's. Also here, found while checking what an upgrade actually runs: the connected update steps ran `flask db upgrade` and stopped. Per-plugin Alembic chains (ADR-008) are not part of that, so a connected site taking an image with a bumped plugin migration ran the core chain and silently skipped every plugin chain. The air-gap stack had it right all along. Both commands are in Step 9 now, plus a `db current` check against `db heads`. |
||
|
cproudlock
|
375dd3fb9d |
Copy the plugin staging script into the build that needs it
`npm run build` fires a `prebuild` hook that runs `node ../scripts/stage-frontend.mjs`, the script that copies each plugin's frontend into the Vite tree and codegens routes.gen.js. The Dockerfile's frontend stage copied `frontend/` alone and flattened it to the stage root, so `../scripts` resolved to `/scripts`, which does not exist. Every image build since the staging script landed has died there on `Cannot find module`. The stage now keeps the repo-relative layout - frontend/, scripts/ and plugins/ as siblings under /build - because the script resolves its repo root as its own directory's parent. The final stage copies from /build/frontend/dist to match. Verified with a real build, not by inspection: 16 plugin frontends staged, vite produced dist, and the finished image carries frontend/dist with 241 assets. This also unblocks SITE_PLUGINS lean image builds, which until now only scripts/build-site.sh could do. |
||
| 11f3d00a04 |
Installer prerequisites: REQ-D through REQ-G
REQ-D: restore waitress and tzdata to requirements.in. They existed ONLY in the
generated requirements.txt (hand-added in
|
|||
|
cproudlock
|
d178726687 |
ADR-013 Phase 0: plugin lifecycle groundwork
Additive, zero-risk-to-running-sites prep for the plugin catalog. No distribution or lean-build behavior yet; fixes latent bugs and adds the declarative + validate tooling later phases build on. Fixes: - upgrade_all_plugins iterates registry.get_all(); only adopted plugins are migrated. Removes the phantom hasattr(registry, 'list_installed') probe that always fell through to migrating every folder on disk (unadopted DDL ran with full DB rights on every deploy). - Reverse-dependency checks on uninstall/disable read dependencies from the manifest on disk via _installed_dependents, so an installed-but-unloaded or disabled dependent is counted. Uninstall blocks on any installed dependent; disable blocks on an enabled dependent. - _sort_by_dependencies detects a dependency cycle (back edge in the DFS) and raises PluginDependencyError instead of looping or dropping a plugin. New: - flask plugin validate <name>: manifest loads + name match, manifest-schema check, core_version admits the framework contract, declared dependencies exist on disk. No new dependency (lightweight checker); schema ships in the package at shopdb/plugins/manifest_schema.json (docs/ is stripped on publish). The check caught that provides is an object, not an array. - flask plugin apply-profile <file>: declarative install AND enable of a chosen plugin set plus its hard-dependency closure, in dependency order, idempotent. Replaces the hand-ordered runbook sequences that could enable a plugin that was never installed. deploy/site-profile.example.json template. - Dockerfile header corrected (all 13 catalog plugins, not "eleven core"). 10 new lifecycle tests (reverse-deps from disk, cycle detection, upgrade-all scope, profile closure, schema, all 13 manifests match schema). 1018 pass, naming green. |
||
|
cproudlock
|
cd353b6432 |
Review safe-polish: docs accuracy, dead imports, no-emoji, geenforce robustness
From the full multi-agent review (0 high, 7 medium, 17 low findings). Applies the mechanical, low-risk items; design/policy findings left for a decision. Docs accuracy: CLAUDE.md contract 0.10.0 -> 0.11.0 and both stale Alembic head citations -> 7d24_customfield_searchable / 31 migrations; Dockerfile bundled- plugin comment fixed (drop nonexistent "equipment", add machines + measuringtools, count eleven). Style/naming (LOCKED rules): remove a CSS-escaped pushpin emoji before location search results (no-emoji policy); rename ManifestEditor shareRoot -> shareroot (variable mirrors the API field verbatim). Dead code: remove confirmed-unused imports across ~20 modules (require_role/ require_permission scaffold residue, stray db/Vendor/Model/current_user/Optional/ error_response); drop unused build_scope import + a stale GEENFORCE_API_KEY docstring clause in geenforce. Migration files left untouched. Correctness: geenforce ingest robustness - record_enforcement_report now 400s on a non-dict counts / non-list results instead of 500; _apply_app_link ignores a non-numeric appid per its docstring instead of 500. Regression tests added. Backend query.get sweep finished: auth.py refresh -> db.session.get (last one). 910 backend tests pass; pyflakes clean; naming green; frontend build green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
b8c22244a1 |
Multi-site distribution readiness: settings-driven site config, security closeout, release engineering, v0.5.0
Make the app distributable to other GE Aerospace sites (one self-hosted
instance per site, ADR-004). GE values remain the shipped defaults; every
site-specific behavior is now a Setting an admin can change in the UI.
Settings-driven site config:
- Branding: site/QR/badge logos, favicon, primary color (upload endpoints
mirror the map-blueprint pattern; new Settings > Branding section).
- ServiceNow: search/incident/change URL templates ({ticket}), ticket
prefixes, enable toggle. Defaults point at the current
geaerospaceqa.service-now.com global search. Disabled = plain-text tickets.
- Employee-id regex (employeeid_pattern), printer hostname template,
QR label targets (qr_target_printer / qr_target_usb, blank = asset page,
else URL template with placeholders), usb_label_style (barcode|qr).
- West Jefferson floor-plan PNGs removed from the tree; generic placeholder
ships as the map default and sites upload their own blueprint.
Security closeout:
- dashboarddefaults writes now require admin.
- Collector: generic error messages (no str(exc) leak); API key accepted
via X-API-Key header only (BREAKING: querystring api_key removed).
- IP-based login rate limiting (AUTH_RATELIMIT_* knobs) atop account lockout.
- Setting.set() creation race fixed (IntegrityError retry).
Release engineering and docs:
- __version__ 0.5.0 (distinct from __contract_version__, ADR-007),
CHANGELOG.md, Gitea Actions CI config, frontend version aligned.
- One wizard-first install story across README/DEPLOY; new CONFIG.md,
UPGRADE.md, BACKUP-RESTORE.md; CLAUDE.md and ROADMAP de-staled.
- Dockerfile multi-stage build now bundles the frontend; compose binds
MySQL to 127.0.0.1; stale database/schema.sql and one-off SQL removed.
Debt and fixes:
- .query.get() -> db.session.get() sweep; datetime.utcnow() removed
(naive-UTC via timezone-aware now); users.py on authz decorators.
- Fixed 4 stale tests (slides feed shape, shopfloor splitperemployee,
plugin contract purity) and the USB label page field mapping (both usb
modes emit the cmmc shape: device_id/device_desc).
- Health endpoint reports the real version.
248 tests pass; naming/style check green; frontend builds; fresh-DB
flask db upgrade + seeds verified; QR targets verified by decoding
rendered codes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
cproudlock
|
06c16bde8a |
Phase 5: Alembic baseline, per-site deploy, ADRs to docs/adr
Migration runner ready and a sister site can deploy from a clean
checkout with one .env file.
ADRs relocated (migrations/adr/ -> docs/adr/):
- migrations/ is now Alembic territory, not docs.
- All cross-references updated: CLAUDE.md, docs/PLUGIN-HOOKS.md,
docs/PLUGIN-QUICKSTART.md.
Alembic initialized (migrations/):
- env.py, script.py.mako, alembic.ini copied from Flask-Migrate
templates so `flask db migrate` and `flask db upgrade` work without
a one-time `flask db init` (which would clash with the existing
migrations/ directory).
- Baseline migration generated via autogenerate, captures all 47
tables (core models + 6 plugins) as the upgrade target. Ready for
per-site `flask db upgrade` from an empty schema.
Deploy artifacts:
- Dockerfile: python:3.12-slim base, gunicorn server, non-root user,
healthcheck against /api/auth/login. Single image bundles all six
plugins; sites enable via `flask plugin install <name>`.
- docker-compose.yml: MySQL 8 + API container, healthcheck-gated
startup, env-driven secrets that fail loud on missing values
(`${SECRET_KEY:?}` form).
- .env.example: full env-var inventory with comments. Calls out
required vs optional. Matches what ProductionConfig.validate
enforces.
docs/DEPLOY.md:
- Step-by-step per-site runbook: clone, configure .env, bring up
stack, run migrations, seed reference data, install plugins,
create admin, front with TLS, backups, updates.
- Common-issues table.
- Cross-links to ADR-004 (per-site rationale), ADR-003 (plugin
distribution), and the config source.
Skills:
- migrating-asset-schema: Alembic + one-shot data migration policy.
Rules: additive first, renames are three steps, destructive ops
need rollback, equipment migration filter per ADR-001 + ADR-005.
- hardening-flask-config: production validation, CORS allowlist
policy, JWT cookie hardening, per-site deploy isolation per ADR-004.
CLAUDE.md updated to reflect the post-Phase-5 state. No tests added
this commit; the Alembic baseline is exercised by the existing
db.create_all-based test suite (tests do not touch the migration
runner; that's by design until per-plugin migrations land).
Test count unchanged: 101 passing.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
