The stale-backup card could not be built as designed, and the reason is more
important than the card. Dedup means an unchanged configuration writes no
revision, so collectedat moves only on a CHANGE. A machine stable for six
months has a six-month-old newest revision and is perfectly healthy. Keying a
staleness card on revision age would have flagged most of the fleet - exactly
the noise that makes a board worth ignoring.
Underneath that: ShopDB could not distinguish those cases at all. On a no-op
the server returned "unchanged" and wrote nothing, so "we checked yesterday and
it matched" was discarded. That fact is the one thing a backup system must be
able to prove, and the only record of it was a line in a log file on the PC.
lastseenat records the check rather than the change. Touched on every matching
post including the no-op; set on creation, since a new revision has by
definition just been seen; backfilled from collectedat or createdat so existing
rows start from the last moment the config can be PROVEN current, rather than
from now - claiming a check that never happened would be worse than silence.
The card keys on it, one row per CHAIN rather than per asset: a machine with
two part markers can have one still reporting while the other stopped, and a
per-asset view would report the machine as fine. It stays deliberately silent
about assets never backed up, because whether one SHOULD be is a question only
the manifest can answer, and guessing would list a hundred healthy machines.
The rule lives in services/staleness.py rather than the route, so it is
testable without an auth layer in the way - the same split retention.py uses.
Threshold is backups_staledays, default 3, and 0 disables the card.
Rows ran in display-style order and then alphabetically, so what led the screen
was an accident of styling and the alphabet - a new type called Awareness
landed above Recertification for no better reason than the letter A.
Each type now carries a board position, lowest first, set on the Notification
Types page. The migration seeds Recognition at 10 and Recertification at 20 and
leaves everything else at 100, so an existing board keeps the order sites
already expect. Steps of ten leave room to slot a row in without renumbering
the rest.
A row shared by several types sits wherever its earliest-ordered type puts it,
so a category moves as a unit.
Adds a kind-pluggable backups plugin. Configuration captured from a PC is
filed against the MACHINE it controls, with a revision history and download
back to the native format.
NTLARS/DNC is the first kind. Settings live in the controlling PC's registry
but describe the machine, so revisions attach to the machine's asset and carry
no foreign key to the PC: history survives a PC being replaced or deleted, and
sourcehostname records the handover.
Storage splits by kind. Parseable kinds store a dialect-neutral JSON
projection in ShopDB and re-render on download; opaque vendor formats (part
marker and similar) keep their bytes on the SFLD share with ShopDB holding
metadata and the UNC pointer.
Two .reg dialects exist in the wild: NTLARS's own Save... export omits the
WOW6432Node path segment, scripted exports include it. Parsing strips whichever
root matched, so a stored revision commits to neither and download offers both
(NTLARS Load... by default, WOW6432Node for direct reg import). Getting this
backwards is silent, so the dedup hash deliberately excludes sourcedialect and
both dialects of one config dedup to a single revision.
Dedup is load-bearing: the collector runs every GE-Enforce cycle across the
fleet, so a revision is inserted only when the content hash differs from that
asset's latest for that kind.
A freshly imaged PC opens NTLARS with a blank General tab. Recording that would
make an empty config the newest revision exactly when someone needs the last
good one, so a blank MachineNo is rejected rather than accepted as a change.
Two of the 320 known-good backups on the share already have that shape.
DNC Info card summarises the latest revision on the machine page: General
(Cnc, NcIF, HostType), eFocas, Serial, NTSHR when populated (only 18 of 147
machines), and MARK when the machine is a marker. MARK is gated on Cnc=MARKER
or the ShopDB machine type, not on the MARK key having content: MARK carries
serial defaults on 145 of 147 machines and DncPatterns reads YES on 103
including ordinary lathes, so neither identifies a marker.
The info card is owned by the kind (BackupKind.infopanel/buildinfo) and served
by a generic endpoint, so the expected successor to DNC ships its own card by
adding a class rather than changing the plugin or the panel wiring.
Also: schedule and retention settings with a prune that never drops the newest
or the oldest revision, and scripts/import_ntlars_backups.py to seed history
from the existing per-machine .reg files (144 of 147 resolve to assets).
Codec verified against all 320 real backups: round-trips clean through both
dialects. Bay-side generation verified on Windows against reg.exe export.
The shopfloor board grouped cards by display style alone, so every type set to
grid landed inside the Recertification row and every carousel type inside
Recognition's - under a heading naming somebody else's type. Setting Awareness
to grid put awareness messages under "Recertification Required".
Each type now gets a row of its own, titled by its own name, and rotation state
is per row: two carousel rows advance on their own indexes instead of sharing
one counter, and two grid rows page independently.
For the other direction there is notificationtypes.boardcategory. Types sharing
a category share one row under the category name, so Change, Awareness and
Incident can sit together while Recognition and Recertification keep their own.
Blank - the default - means a row of its own. The category is part of the
grouping key along with the display style, since a category cannot merge a
banner with a row of tiles.
A card that names no employee now renders its message as the tile or card,
rather than a placeholder face above a blank name, which is what a grid type
like Awareness looked like before.
The layout fingerprint that makes open kiosks reload now covers the category
and the grace window, so a re-grouped board reaches screens that are already up.
The shopfloor feed kept every ended notification up for a hardcoded 30 minutes,
flagged resolved. A card with an 8:00 end time was still on the board at 8:29,
which reads as an expiry that did not work - and in the carousel, grid and
banner sections it read that way with no visual sign at all, since only the
standard cards render the resolved state.
The tail is now notificationtypes.gracewindowminutes, set per type on the
Notification Types page and defaulting to 0, so an end time means what it says.
A type whose cards are worth acknowledging after they clear - an incident, say
- opts into a tail, and only that type's cards get one.
The feed widens its query to the largest configured tail and then holds each
row to its own type's window. That keeps one portable query rather than a
per-type interval expression in SQL, and with every type at 0 it collapses to
"still showing".
Also fixes resolved serializing as null rather than false for a card with no
end time, which the and-chain produced.
Three faults around vendor-model photos, found while looking at why an uploaded
image did not appear.
Saving a model was blocked after uploading a photo. The Image URL field was
type="url", and an upload sets it to an application path such as
/api/models/image/model-120.png. Native url validation demands an absolute URL
with a scheme, so the browser refused to submit the form with "Please enter a
URL" for a value the page had just written itself. The field is now type="text",
which is what it always needed to be: it holds either a full web address or a
path on this server. documentationurl stays type="url".
The upload button did not appear when adding a model, only when editing one.
That was deliberate - the photo is stored as model-<id>.<ext>, so it cannot be
sent before the record has an id - but it reads as a missing feature, and the
hint explaining it was easy to miss. A photo chosen while creating is now held
and uploaded as soon as the model is saved, and it is dropped if the dialog is
cancelled, so it cannot land on the next model created in the same session.
Network devices could never show a photo. NetworkDeviceDetail.vue binds its hero
image to networkdevice.imageurl, but networkdevices carried only vendorid, with
no link to a catalog model, so nothing could populate it - a feature that looked
present and could not work. Machines, PCs and printers have carried
modelnumberid since July. This adds the same column and relationship, the
to_dict branch that exposes modelname and imageurl, the field on the API, and a
Model selector on the form so the link can actually be set.
The migration is guarded the same way employees0002photo is: on a fresh database
the tables come from the SQLAlchemy models, which already declare the column, so
an unconditional add fails with "duplicate column name". The foreign key is
created only on databases that can add one by ALTER; routing it through
batch_alter_table made Alembic's column sort raise "Circular dependency
detected" on the fresh-database test.
Deploying this needs `flask db upgrade` and `flask plugin upgrade-all` on the
server, not just a file copy.
Get GE-Enforce closer to running on credential-less Intune/Entra display PCs
that pull manifest + payloads over HTTPS instead of SMB.
Server (plugins/geenforce/api/routes.py):
- Rate-limit + 512MB served-size ceiling on GET /payload/<sha256> (reuses the
login limiter's cache pattern, config-overridable via GEENFORCE_PAYLOAD_*).
- New tests: payload hardening, manifestblobs model-vs-migration parity, and a
report-contract test locking the lowercase per-entry report keys.
PS client (plugins/geenforce/client/):
- Fix New-ShopdbReport per-entry key casing to lowercase (name/action/selfhealed/
exitcode/message) to match what the server reads; the engine emits PascalCase.
- Enforce TLS 1.2 in the network functions.
- Fetch + merge the fleet-wide common scope alongside the pctype scope
(pctype wins on conflict; -NoCommon opt-out).
- Normalize whatever the engine returns into a well-formed summary.
- Make the empty-cache fail-safe observable: event-log entry + report ping
instead of a silent exit 0.
Manifest (plugins/geenforce/seed_display_scope.py + docs/GE-ENFORCE-DISPLAY.md):
- Seed a gea-shopfloor-display scope: 4 Edge kiosk drift-heal registry entries
+ 1 data-driven dispatcher (Dashboard/Lobby/3DPrintRoom via display-type.txt).
Kiosk EXEs stay image-baked; the manifest heals policy/config drift only.
- Documents the common SMB-payload audit (entries needing http/inline before a
share-less display can inherit common).
Migration registry (shopdb/plugins/alembic_template.py + test):
- Register the pre-existing manifestblobs and the new printersupplyalerts tables
in PLUGIN_TABLE_OWNERS; update EXPECTED_HEAD_REVISION for geenforce (0002blobs),
printers (0002supplyalerts), and printedparts (0004txnrev) which had drifted.
The gage lab assigns real WJRP asset numbers, so identity splits: the
internal itemcode stays auto-minted and a new optional unique
gagelabtag (migration 0003) carries the lab's number - settable on
create/edit, searchable, and resolved by the kiosk for scans and bare
keypad digits against the numeric tail of either identifier
(unique-match only). The print-files table becomes stacked revision
cards - filename with rev/current badges, one meta line, delete pinned
right - ending the horizontal scroll in that column.
printeditemfiles lands as the plugin's first incremental migration
(0002 on the plugin chain - the ADR-008 payoff). Revisions are
append-only per item: upload assigns the next number, records the
uploader from the JWT, enforces an extension allowlist and a 100 MB
cap; download serves the original filename; a permission-gated delete
covers wrong-file mistakes. The detail page gains the revision table
with a current badge. Unique storedfilename is sized 191 so the index
fits MySQL's 767-byte prefix - the per-plugin chain does not apply the
core env's ROW_FORMAT hook.
Alert recipients gain roles: Role joins the 0.13.0 surface, a role
picker on the settings page, and every active member of the selected
roles is folded into the deduped recipient list.
PrintedItem (catalog: code, name, image, cached quantityonhand,
per-item threshold, bin) and PrintedItemTransaction (the ledger:
signed quantity change attributed to a badge-resolved employee).
Both registered in PLUGIN_TABLE_OWNERS; 0001 is a post-cutover real
baseline. The migration-guard test learns the new expected head.
Routes are a placeholder ping until the next stage - the scaffold's
list route imported the deleted scaffold model, which surfaces as an
empty 'Migration error' because the alembic env imports the models
package.
From the database review (verdict: sound-with-minor-issues). Applies the
actionable findings.
Redundant indexes: five non-unique secondary indexes duplicated a named idx_*
or a unique index on the same column - ix_communications_assetid,
ix_computers_hostname, ix_networkdevices_hostname, ix_printers_hostname (each
shadowing an idx_*), and idx_usb_serial (shadowing the serialnumber unique
index). Removed the redundant index source from the models (column index=True /
the extra db.Index) and added core migration 7d25 dropping the live duplicates.
The unique ix_*_assetid indexes are kept (they enforce assetid uniqueness).
Dead column: usbcheckouts.machineid was a NOT NULL soft-ref to the retired
machines table storing sentinel 0 (ADR-001). Dropped from the model + the
machineid=0 literal in selfhosted checkout; usb plugin migration 0002 drops it
live (downgrade restores it default 0).
Index: notifications.businessunitid (filtered by the shopfloor feed) was
unindexed; added index=True + notifications migration 0002.
CI: new migrations-mysql job proves the real multi-site deploy path - fresh
`flask db upgrade` + per-plugin install on utf8mb4 MySQL from empty, asserting
table count + charset and a clean second-run no-op. The pytest suite only
exercises SQLite create_all(), so a regression in the Alembic chain on MySQL
would otherwise ship undetected.
Verified: fresh core upgrade on a scratch utf8mb4 MySQL builds clean + no-op on
rerun (redundant indexes absent, unique assetid kept); plugin migrations applied
+ verified on the dev DB (machineid gone, bu index present). 953 backend tests
pass; naming + pyflakes green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
First execution phases of docs/proposals/ge-enforce-plugin.md. The GE-Enforce
manifest becomes shopdb data.
P0 scaffold: new geenforce plugin (api_prefix /api/geenforce, default_enabled
false, core_version >=0.7.0). Registered in PLUGIN_TABLE_OWNERS (ADR-008); its
0001 baseline really creates the tables.
P1a model: one wide manifestentries table + entrytype discriminator (not STI,
not JSON blob), manifestscopes (UNIQUE scopename+phase), the three multi-value
filter child tables, inusechecks + processes, immutable manifestpublishedversions
(frozen rendered JSON), manifestpayloads (inline, capped), pctypealiases
(mirror of the engine lib's alias graph). regvalue stored as its raw JSON
literal so DWord typing survives.
P1c importer + exporter: parse common + gea-shopfloor-* + preinstall.json into
draft rows and rebuild the JSON verbatim from rows in sortorder.
P1d parity harness (GATE A): filters.py mirrors the engine's four filter
functions + alias graph; parity.py proves import+export is behaviorally lossless
(field-identical + same-entries-fire across 18 machine-profile fixtures) WITHOUT
byte-diffing. Verified PASS against all 11 real reference manifests (64 entries)
and a synthetic site-neutral fixture covering every type/filter (the CI gate).
First slice (gea-shopfloor-cmm shape): service layer (import/publish/rollback/
export-to-share), CLI (parity, import-share, publish, export-share), and the
client endpoint GET /api/geenforce/manifest serving the current published
snapshot (never the draft) with ETag/304. Split permissions
geenforce.manage/publish/fetch. Tests prove import->publish->serve, draft edits
never change served bytes, publish+rollback, and auth (401 unauth/wrong-scope).
Contract 0.11.0: added service_token_authorized(scope) to shopdb.api so plugin
service endpoints authorize a scoped managed token without importing core token
internals. Documented in PLUGIN-HOOKS.md.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Model photos: upload/replace/delete on /api/models/<id>/image (admin),
stored under instance/modelimages/ with a public serve route; thumbnail
plus Upload/Replace/Remove controls in the Models settings modal; the
URL field remains as a manual alternative.
Employee photos, mode-aware: self-hosted directory employees get
upload/replace/delete (photo-<sso> under instance/employeephotos/,
employees plugin migration 0002); external directory mode passes the
HR-supplied picture URL through read-only (writes 409). One resolver
feeds both consumers - the shopfloor recognition/recert kiosk cards and
the employee detail hero - in either mode.
Navigation fix: router-view is keyed on route path, so following a
relationship link between two assets of the same type (machine ->
dualpath machine) reloads the page instead of showing stale content;
query-only URL changes still avoid a remount.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The equipment plugin is now the machines plugin, ending the UI-vs-code
vocabulary split while the contract is pre-1.0 and nothing external
depends on the old names.
- plugins/equipment -> plugins/machines: manifest, class, /api/machines,
machines.* permissions, registry key (with an auto-migrating load shim
for existing installs).
- Tables: equipment -> machines (equipmentid -> machineid) and
equipmenttypes -> machinetypes, renamed in the plugin's own migration
chain (machines0002rename), idempotent for both upgrading and fresh
installs.
- The legacy core machinetypes lookup actually types the vendor MODELS
catalog, so it is renamed losslessly to modeltypes
(models.modeltypeid, /api/modeltypes, Model Types settings page)
rather than collapsed, freeing the machinetypes name. Core migration
7d17_machines_rename also flips data in place: assettypes row
equipment -> machine, auditlog entitytype, identifier_/search_
settings keys, permission rows, and renames alembic_version_equipment.
- Frontend: machinesApi/modeltypesApi, item.machine response shape,
assettype value compares 'equipment' -> 'machine' (map, search,
custom fields, relationships), routes machines.js with plugin gating
retagged, /print/machine-badge, Machine Types (subtypes) and Model
Types (catalog) settings pages, machines-by-type report id.
- Docs swept; ADRs left as history per the authoring rule.
Upgrade: flask db upgrade then flask plugin upgrade-all.
Verified: dev DB flipped live (262 machines, 35 modeltypes, 95 models
retyped, zero equipment tables remain); fresh scratch-MySQL install
produces the new names; 341 tests green; naming/style green; frontend
builds; live E2E on machines list/detail, PC relationships, map,
reports, and both settings pages.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Framework:
- Per-plugin Alembic migration chains (ADR-008): every bundled plugin
carries its own chain with a stamp-only anchor at the ownership cutover;
new plugin schema lands in plugins/<name>/migrations/, never the core
chain. Deploys add flask plugin upgrade-all. Fixed a latent bug in the
shared alembic template (engine URL resolution) and taught the metadata
filter to include FK-referenced core tables.
- Frontend plugin route gating (ADR-009): plugin routes carry meta.plugin;
a disabled plugin's pages redirect to the dashboard via a cached,
fail-open check against the new public GET /api/plugins/enabled.
- get_reports() plugin hook (contract 0.5.0 -> 0.6.0): plugins contribute
report cards; warranty and toner cards moved off the hardcoded list.
Reports:
- Hub grouped by category with search; inline reports render at the top,
are URL-backed (?report=id, back-button and deep links work), expose
their server-side filter params as controls, and export CSV. Warranty
and Toner pages gained CSV export.
- Deleted the dead legacy Warranty Status report (always-zero buckets
from a retired column).
Theming and fonts:
- Inter (variable) bundled locally via @fontsource, replacing the Google
Fonts Roboto import - air-gapped installs now render correctly; tables
use tabular numerals.
- Optional brand_primary_dark_color, brand_accent_color,
brand_sidebar_color settings applied to CSS vars at bootstrap.
USB frontend repair (views were reading a dead legacy shape):
- List/detail/form and the employee profile USB panels remapped to the
real API shape (device_id/device_desc/checkinoutlog); employee panels
now use /usb/checkouts endpoints; external-mode /usb/checkouts/active
honors the badge filter; dead client methods pruned.
Also: warranties list page no longer requires login (matches app
convention); collector doc rewritten with a GE-Enforce integration guide
and paste-ready PowerShell reporter; ADR index and CHANGELOG updated.
Verified: 323 tests pass, naming/style green, frontend builds, plugin
migration dry-run green on scratch MySQL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The core chain already owns and reproduces the full bundled schema (deploys run
`flask db upgrade` only). The per-plugin Alembic baselines duplicated those
tables, so `flask plugin upgrade-all` would conflict - a footgun. Remove the 6
bundled plugin migration dirs; the per-plugin Alembic helpers
(alembic_template, PluginMigrationRunner) remain for external/filesystem
plugins. upgrade-all now cleanly no-ops for bundled plugins.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each of the six bundled plugins (computers, equipment, network,
notifications, printers, usb) now has its own Alembic chain with a
baseline migration. Sister sites adopting one of these plugins can
manage its schema via `flask plugin migrate <name>` instead of relying
on db.create_all to bootstrap everything.
Existing single-site deploys that bootstrap via db.create_all continue
to work unchanged. The chains coexist; the bootstrap path stays the
operator's choice.
Framework
- shopdb/plugins/alembic_template.py: shared env.py logic + helpers.
PLUGIN_TABLE_OWNERS pins which tables belong to which plugin (explicit
registry, not import-side-effect). _get_plugin_metadata filters
db.metadata to only the named plugin's tables. create_plugin_tables /
drop_plugin_tables emit DDL via SQLAlchemy CreateTable so the table
definitions stay sourced from the models, not duplicated.
- shopdb/plugins/__init__.py: PluginManager.upgrade_all_plugins() runs
pending migrations across every discovered plugin and returns a status
dict. Idempotent (Alembic skips applied revisions).
CLI
- `flask plugin upgrade-all` runs pending migrations for every plugin.
Used on a fresh deploy after the core schema is in place.
Per-plugin scaffolding
- plugins/{computers,equipment,network,notifications,printers,usb}/
migrations/{alembic.ini, env.py, script.py.mako, versions/0001_baseline.py}
- Each env.py is a 5-line shim that sets PLUGIN_NAME and delegates to
the shared template. Each 0001_baseline calls create_plugin_tables(name)
/ drop_plugin_tables(name); no duplication of column definitions.
Tests
- tests/test_plugin_migrations.py (18 cases): every bundled plugin has
an entry in PLUGIN_TABLE_OWNERS, has the on-disk Alembic scaffolding,
and the filtered MetaData contains every owned table (catches drift
between the template's table list and what the models declare).
- 129 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
