65f53e91cd88d935b54fcb7d208df8fe496fb6ff
13 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
cproudlock
|
fca775c737 |
backups plugin: per-asset config backups with revision history
Adds a kind-pluggable backups plugin. Configuration captured from a PC is filed against the MACHINE it controls, with a revision history and download back to the native format. NTLARS/DNC is the first kind. Settings live in the controlling PC's registry but describe the machine, so revisions attach to the machine's asset and carry no foreign key to the PC: history survives a PC being replaced or deleted, and sourcehostname records the handover. Storage splits by kind. Parseable kinds store a dialect-neutral JSON projection in ShopDB and re-render on download; opaque vendor formats (part marker and similar) keep their bytes on the SFLD share with ShopDB holding metadata and the UNC pointer. Two .reg dialects exist in the wild: NTLARS's own Save... export omits the WOW6432Node path segment, scripted exports include it. Parsing strips whichever root matched, so a stored revision commits to neither and download offers both (NTLARS Load... by default, WOW6432Node for direct reg import). Getting this backwards is silent, so the dedup hash deliberately excludes sourcedialect and both dialects of one config dedup to a single revision. Dedup is load-bearing: the collector runs every GE-Enforce cycle across the fleet, so a revision is inserted only when the content hash differs from that asset's latest for that kind. A freshly imaged PC opens NTLARS with a blank General tab. Recording that would make an empty config the newest revision exactly when someone needs the last good one, so a blank MachineNo is rejected rather than accepted as a change. Two of the 320 known-good backups on the share already have that shape. DNC Info card summarises the latest revision on the machine page: General (Cnc, NcIF, HostType), eFocas, Serial, NTSHR when populated (only 18 of 147 machines), and MARK when the machine is a marker. MARK is gated on Cnc=MARKER or the ShopDB machine type, not on the MARK key having content: MARK carries serial defaults on 145 of 147 machines and DncPatterns reads YES on 103 including ordinary lathes, so neither identifies a marker. The info card is owned by the kind (BackupKind.infopanel/buildinfo) and served by a generic endpoint, so the expected successor to DNC ships its own card by adding a class rather than changing the plugin or the panel wiring. Also: schedule and retention settings with a prune that never drops the newest or the oldest revision, and scripts/import_ntlars_backups.py to seed history from the existing per-machine .reg files (144 of 147 resolve to assets). Codec verified against all 320 real backups: round-trips clean through both dialects. Bay-side generation verified on Windows against reg.exe export. |
||
|
cproudlock
|
2c415a1712 |
fix(installer): correct a false security claim, and clear the should-fix list
CLIENT IP / SPOOFABILITY. docs/geenforce-api-cutover.md claimed that removing the IIS rewrite rule made the allowlist fail closed and that it does NOT become spoofable. The opposite is true. IIS never sets X-Forwarded-For on its own; the rule is the only thing that does. Remove it and IIS still forwards whatever X-Forwarded-For the CALLER sent, waitress trusts it because it arrives from 127.0.0.1, and remote_addr becomes attacker-controlled - so a token-less caller can fetch manifests from anywhere on the network. The document and the _trusted_client_ip docstring now say so, waitress runs with --trusted-proxy-count=1, and stage 5 checks the rule is actually live rather than assuming it. The wizard question is rephrased to something an operator can verify with their network team instead of guessing at. NON-ASCII. The style gate only ever checked .py/.vue/.js/.ts, so documentation accumulated em-dashes, arrows and box-drawing characters against this repo's own convention - including in files added this week. Cleaned, and the gate now uses INCLUDES_ALL so Markdown, JSON and YAML are covered. PLUGIN DEFAULTS. The wizard pre-ticked measuringtools and printedparts, both of which ship default_enabled=false, so every site taking the defaults installed and enabled them against their manifests. Inno has no JSON parser so the list must be hardcoded, but tests/test_installer_defaults.py now fails when it drifts. UPGRADES. The payload copy merges, so a plugin dropped from a site's profile kept its code forever - which defeats a lean build and leaves core's optional-import guards succeeding for a plugin the site no longer has. Stale plugin directories are now deregistered and removed before the copy. add-plugin used 'plugin install', which for the five default_enabled=false plugins left them installed but DISABLED - and printed a green success line anyway. It now goes through apply-profile, and the success line is gated on the exit code. Invoke-Flask records its own exit status, because $LASTEXITCODE keeps a stale value when flask.exe is missing and no native command runs. CHARSET. The utf8mb4 compiler hook lived inline in migrations/env.py, so it covered the CORE chain only: plugin baselines inherited the server default, which on a latin1 server means two charsets in one database. It is now shopdb/utils/mysql_charset.py, imported by both, and preflight reports the database's default charset. BACKUP HONESTY. The dump was described as 'all of your asset data'. Uploaded branding and floor-map images live in instance\ on disk, not in the database, so a restore from the .sql alone comes back with no map. backup now archives instance\ alongside it and says both are needed. VERSIONING. AppVersion was hardcoded at 0.9.0 while the product, the frontend and the newest tag said 0.7.0 - and 0.9.0 collides with a retired contract version. Both builders now generate version.iss from shopdb/__init__.py. Smaller: rollback overwrites .env before deleting it, as uninstall already did; appcmd unlocks are scoped to this site's location rather than server-wide, with the wide unlock as a fallback; DEVELOPMENT-SETUP says Python 3.14; the README plugin list gains printedparts; prune-schema --force is documented as first-provisioning-only; HTTPS is documented as not-the-default with the steps to add it; the DBA SQL is on the wizard's database page; the features page says unticking does not remove an installed feature; and the installer README states that bundle-lock cannot vouch for the exe itself - that needs signing or an out-of-band hash, neither of which is wired up. |
||
|
cproudlock
|
9d65ef103d |
geenforce: display-readiness batch (server hardening, PS client wiring, display scope)
Get GE-Enforce closer to running on credential-less Intune/Entra display PCs that pull manifest + payloads over HTTPS instead of SMB. Server (plugins/geenforce/api/routes.py): - Rate-limit + 512MB served-size ceiling on GET /payload/<sha256> (reuses the login limiter's cache pattern, config-overridable via GEENFORCE_PAYLOAD_*). - New tests: payload hardening, manifestblobs model-vs-migration parity, and a report-contract test locking the lowercase per-entry report keys. PS client (plugins/geenforce/client/): - Fix New-ShopdbReport per-entry key casing to lowercase (name/action/selfhealed/ exitcode/message) to match what the server reads; the engine emits PascalCase. - Enforce TLS 1.2 in the network functions. - Fetch + merge the fleet-wide common scope alongside the pctype scope (pctype wins on conflict; -NoCommon opt-out). - Normalize whatever the engine returns into a well-formed summary. - Make the empty-cache fail-safe observable: event-log entry + report ping instead of a silent exit 0. Manifest (plugins/geenforce/seed_display_scope.py + docs/GE-ENFORCE-DISPLAY.md): - Seed a gea-shopfloor-display scope: 4 Edge kiosk drift-heal registry entries + 1 data-driven dispatcher (Dashboard/Lobby/3DPrintRoom via display-type.txt). Kiosk EXEs stay image-baked; the manifest heals policy/config drift only. - Documents the common SMB-payload audit (entries needing http/inline before a share-less display can inherit common). Migration registry (shopdb/plugins/alembic_template.py + test): - Register the pre-existing manifestblobs and the new printersupplyalerts tables in PLUGIN_TABLE_OWNERS; update EXPECTED_HEAD_REVISION for geenforce (0002blobs), printers (0002supplyalerts), and printedparts (0004txnrev) which had drifted. |
||
|
cproudlock
|
42ca8d75c3 |
ADR-014: schema-lean per-site (investigation + idempotent create_plugin_tables)
Cross-plugin FK blocker ADR-013 cited is already resolved: the FKs into machines were held only by dead legacy tables (machinerelationships, printerdata, installedapps, communications.machineid) that existing migrations 7a01/7c01 already drop. No live plugin table hard-FKs another plugin. Schema-lean is unblocked. Enabling change: create_plugin_tables now skips already-existing tables (idempotent) so a plugin anchor can create its tables on a fresh lean install and no-op on a database that has them from the pre-cutover baseline. The load-bearing baseline lift is staged as ADR-014 Phase 2. |
||
|
cproudlock
|
aa4bfcd41c |
printedparts stage 15: print-file revision history + role-based alerts
printeditemfiles lands as the plugin's first incremental migration (0002 on the plugin chain - the ADR-008 payoff). Revisions are append-only per item: upload assigns the next number, records the uploader from the JWT, enforces an extension allowlist and a 100 MB cap; download serves the original filename; a permission-gated delete covers wrong-file mistakes. The detail page gains the revision table with a current badge. Unique storedfilename is sized 191 so the index fits MySQL's 767-byte prefix - the per-plugin chain does not apply the core env's ROW_FORMAT hook. Alert recipients gain roles: Role joins the 0.13.0 surface, a role picker on the settings page, and every active member of the selected roles is folded into the deduped recipient list. |
||
|
cproudlock
|
f5cfac33b4 |
printedparts stage 2: models, real 0001 baseline, tables live
PrintedItem (catalog: code, name, image, cached quantityonhand, per-item threshold, bin) and PrintedItemTransaction (the ledger: signed quantity change attributed to a badge-resolved employee). Both registered in PLUGIN_TABLE_OWNERS; 0001 is a post-cutover real baseline. The migration-guard test learns the new expected head. Routes are a placeholder ping until the next stage - the scaffold's list route imported the deleted scaffold model, which surfaces as an empty 'Migration error' because the alembic env imports the models package. |
||
|
cproudlock
|
6dc31c6149 |
Add GE-Enforce observed-state reporting: receipt + self-heal from PCs
PCs now report enforcement results back to shopdb, closing the desired-vs-observed loop. - POST /api/geenforce/report (geenforce.report service token): each cycle a PC posts the published version it applied, install/skip/fail/filtered counts, and per-entry outcomes. - Two tables: manifestenforcementreports (latest-per-host + history: applied version, enforcer version, counts, derived status ok/selfhealed/failed) and manifestenforcementresults (per entry: action installed/skipped/failed, selfhealed flag, exit code, warning/error message). - RECEIVED: reports carry the applied version; the admin view derives receivedlatest by comparing it to the scope's current published version, so the fleet view shows which PCs picked up an update. - SELF-HEAL: per-entry action captures drift correction (installed when it should already be present) vs skipped (already good) vs failed, with messages. - Admin reads: GET /reports (fleet compliance rollup) and GET /reports/<id> (per-entry detail). New geenforce.report permission. - Tables added to the (undeployed) 0001 baseline; geenforce.post_report is a service-token endpoint so it is exempt from the JWT authz sweep, like the collector blueprint. 8 reporting tests; full suite green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
d85b33bd68 |
Build GE-Enforce manifest-store plugin (P0/P1): model, importer, parity gate
First execution phases of docs/proposals/ge-enforce-plugin.md. The GE-Enforce manifest becomes shopdb data. P0 scaffold: new geenforce plugin (api_prefix /api/geenforce, default_enabled false, core_version >=0.7.0). Registered in PLUGIN_TABLE_OWNERS (ADR-008); its 0001 baseline really creates the tables. P1a model: one wide manifestentries table + entrytype discriminator (not STI, not JSON blob), manifestscopes (UNIQUE scopename+phase), the three multi-value filter child tables, inusechecks + processes, immutable manifestpublishedversions (frozen rendered JSON), manifestpayloads (inline, capped), pctypealiases (mirror of the engine lib's alias graph). regvalue stored as its raw JSON literal so DWord typing survives. P1c importer + exporter: parse common + gea-shopfloor-* + preinstall.json into draft rows and rebuild the JSON verbatim from rows in sortorder. P1d parity harness (GATE A): filters.py mirrors the engine's four filter functions + alias graph; parity.py proves import+export is behaviorally lossless (field-identical + same-entries-fire across 18 machine-profile fixtures) WITHOUT byte-diffing. Verified PASS against all 11 real reference manifests (64 entries) and a synthetic site-neutral fixture covering every type/filter (the CI gate). First slice (gea-shopfloor-cmm shape): service layer (import/publish/rollback/ export-to-share), CLI (parity, import-share, publish, export-share), and the client endpoint GET /api/geenforce/manifest serving the current published snapshot (never the draft) with ETag/304. Split permissions geenforce.manage/publish/fetch. Tests prove import->publish->serve, draft edits never change served bytes, publish+rollback, and auth (401 unauth/wrong-scope). Contract 0.11.0: added service_token_authorized(scope) to shopdb.api so plugin service endpoints authorize a scoped managed token without importing core token internals. Documented in PLUGIN-HOOKS.md. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
48d3160bc5 |
Rename the equipment domain to machines; retype the models catalog (ADR-011)
The equipment plugin is now the machines plugin, ending the UI-vs-code vocabulary split while the contract is pre-1.0 and nothing external depends on the old names. - plugins/equipment -> plugins/machines: manifest, class, /api/machines, machines.* permissions, registry key (with an auto-migrating load shim for existing installs). - Tables: equipment -> machines (equipmentid -> machineid) and equipmenttypes -> machinetypes, renamed in the plugin's own migration chain (machines0002rename), idempotent for both upgrading and fresh installs. - The legacy core machinetypes lookup actually types the vendor MODELS catalog, so it is renamed losslessly to modeltypes (models.modeltypeid, /api/modeltypes, Model Types settings page) rather than collapsed, freeing the machinetypes name. Core migration 7d17_machines_rename also flips data in place: assettypes row equipment -> machine, auditlog entitytype, identifier_/search_ settings keys, permission rows, and renames alembic_version_equipment. - Frontend: machinesApi/modeltypesApi, item.machine response shape, assettype value compares 'equipment' -> 'machine' (map, search, custom fields, relationships), routes machines.js with plugin gating retagged, /print/machine-badge, Machine Types (subtypes) and Model Types (catalog) settings pages, machines-by-type report id. - Docs swept; ADRs left as history per the authoring rule. Upgrade: flask db upgrade then flask plugin upgrade-all. Verified: dev DB flipped live (262 machines, 35 modeltypes, 95 models retyped, zero equipment tables remain); fresh scratch-MySQL install produces the new names; 341 tests green; naming/style green; frontend builds; live E2E on machines list/detail, PC relationships, map, reports, and both settings pages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
cproudlock
|
22e623c1f6 |
Plugin framework maturation, reports overhaul, theming, and USB frontend repair
Framework: - Per-plugin Alembic migration chains (ADR-008): every bundled plugin carries its own chain with a stamp-only anchor at the ownership cutover; new plugin schema lands in plugins/<name>/migrations/, never the core chain. Deploys add flask plugin upgrade-all. Fixed a latent bug in the shared alembic template (engine URL resolution) and taught the metadata filter to include FK-referenced core tables. - Frontend plugin route gating (ADR-009): plugin routes carry meta.plugin; a disabled plugin's pages redirect to the dashboard via a cached, fail-open check against the new public GET /api/plugins/enabled. - get_reports() plugin hook (contract 0.5.0 -> 0.6.0): plugins contribute report cards; warranty and toner cards moved off the hardcoded list. Reports: - Hub grouped by category with search; inline reports render at the top, are URL-backed (?report=id, back-button and deep links work), expose their server-side filter params as controls, and export CSV. Warranty and Toner pages gained CSV export. - Deleted the dead legacy Warranty Status report (always-zero buckets from a retired column). Theming and fonts: - Inter (variable) bundled locally via @fontsource, replacing the Google Fonts Roboto import - air-gapped installs now render correctly; tables use tabular numerals. - Optional brand_primary_dark_color, brand_accent_color, brand_sidebar_color settings applied to CSS vars at bootstrap. USB frontend repair (views were reading a dead legacy shape): - List/detail/form and the employee profile USB panels remapped to the real API shape (device_id/device_desc/checkinoutlog); employee panels now use /usb/checkouts endpoints; external-mode /usb/checkouts/active honors the badge filter; dead client methods pruned. Also: warranties list page no longer requires login (matches app convention); collector doc rewritten with a GE-Enforce integration guide and paste-ready PowerShell reporter; ADR index and CHANGELOG updated. Verified: 323 tests pass, naming/style green, frontend builds, plugin migration dry-run green on scratch MySQL. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
cproudlock
|
b0a0670fcd |
Retire the legacy Machine model (ADR-001 cutover)
The asset/computer model is now the single source of truth. Remove the Machine instance layer end to end: - Delete models Machine, MachineStatus, PCType, MachineRelationship, InstalledApp, PrinterData; keep MachineType (models.machinetypeid still references it). - Delete the /api/machines, /api/statuses, /api/pctypes blueprints and the legacy /api/printers/legacy (PrinterData) blueprint. - Drop the deprecated communications.machineid column and its FK. - Migration 7c01 drops tables machines, machinestatuses, pctypes, machinerelationships, installedapps, printerdata (idempotent). - Fix remaining readers (applications install counts) to ComputerInstalledApp. - Frontend: remove dead machinesApi/statusesApi/pctypesApi wrappers; repoint the PC Types settings page at computer types. 143 tests pass; all asset/computer/dashboard/report/collector endpoints 200. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
0436e8b0af |
Asset-side printer save, status CRUD, real printer types, network fix
- Printers save through the asset blueprint (PUT /printers) instead of the legacy machines API; restrict supply-model picker to printer models. - Asset statuses get full CRUD (PUT/DELETE with in-use guard); canonical set. - Printer types reseeded to a real classification set + list filter. - Equipment accepts gauge/maintenance references. - Fix network list emitting network_device instead of networkdevice (View 404). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
42ae985d72 |
Phase 7B: per-plugin Alembic chains for bundled plugins
Each of the six bundled plugins (computers, equipment, network,
notifications, printers, usb) now has its own Alembic chain with a
baseline migration. Sister sites adopting one of these plugins can
manage its schema via `flask plugin migrate <name>` instead of relying
on db.create_all to bootstrap everything.
Existing single-site deploys that bootstrap via db.create_all continue
to work unchanged. The chains coexist; the bootstrap path stays the
operator's choice.
Framework
- shopdb/plugins/alembic_template.py: shared env.py logic + helpers.
PLUGIN_TABLE_OWNERS pins which tables belong to which plugin (explicit
registry, not import-side-effect). _get_plugin_metadata filters
db.metadata to only the named plugin's tables. create_plugin_tables /
drop_plugin_tables emit DDL via SQLAlchemy CreateTable so the table
definitions stay sourced from the models, not duplicated.
- shopdb/plugins/__init__.py: PluginManager.upgrade_all_plugins() runs
pending migrations across every discovered plugin and returns a status
dict. Idempotent (Alembic skips applied revisions).
CLI
- `flask plugin upgrade-all` runs pending migrations for every plugin.
Used on a fresh deploy after the core schema is in place.
Per-plugin scaffolding
- plugins/{computers,equipment,network,notifications,printers,usb}/
migrations/{alembic.ini, env.py, script.py.mako, versions/0001_baseline.py}
- Each env.py is a 5-line shim that sets PLUGIN_NAME and delegates to
the shared template. Each 0001_baseline calls create_plugin_tables(name)
/ drop_plugin_tables(name); no duplication of column definitions.
Tests
- tests/test_plugin_migrations.py (18 cases): every bundled plugin has
an entry in PLUGIN_TABLE_OWNERS, has the on-disk Alembic scaffolding,
and the filtered MetaData contains every owned table (catches drift
between the template's table list and what the models declare).
- 129 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|