Wave 1 of the remediation plan for the 2026-09-03 feature test run. Two kiosks,
one shape of defect, one critical consequence.
Both kiosks read the state, decided in Python, and then wrote. Nothing stops two
requests interleaving between the read and the write, so both see the same
pre-state and both proceed. Reproduced on MySQL 5.6 over HTTP, five concurrent
takes of 5 from a bin holding 5: all five were accepted and quantityonhand
landed on -20, because each request wrote its own arithmetic. The ledger had
recorded every one of them, so the cached quantity and the ledger disagreed
permanently, which is the number the shop floor actually reads. The USB kiosk
had the same shape: concurrent scans of one drive all succeeded and inserted
several open custody rows, so one drive was recorded out to two people and the
documented 409 never fired.
The comparison now lives in the WHERE clause, where the database settles it at
write time. printedparts._move_quantity issues
UPDATE ... WHERE quantityonhand >= :n and treats rowcount 0 as the refusal,
raising InsufficientStock, which take and adjust turn back into the message they
already gave - carrying the quantity the database holds now, not the stale one
the request read. The low-stock alert derives its before-value from what the
write produced for the same reason. Both USB paths claim the device with
ischeckedout (self-hosted) or status (external cmmc_usb) in the WHERE clause;
the external one claims BEFORE writing the log line, so a refused claim leaves
nothing to undo. Check-in is guarded the same way.
After the fix, the same five concurrent takes: one accepted, four refused,
quantityonhand 0 and the ledger sum 0.
Check-in also closes EVERY open row for the device rather than the newest.
One device can only be out once, so a second open row is corruption, and
closing only the newest left the older ones open forever - the device then read
as returned while its custody history said otherwise.
Separately, usbcheckouts now records who handed a drive back
(checkinsso/checkinname, migration usb0004checkinidentity). Returning for a
colleague is a supported flow that the kiosk asks to confirm, but the row it
wrote attributed the check-in to the HOLDER, so the custody history and the
audit report both named somebody who was not there. Nullable and deliberately
not backfilled: copying the holder's badge into old rows would manufacture
evidence for the exact claim this stops making.
Four tests. Both guards verified by planting: strip the condition out of either
UPDATE and its test fails.
Scope: product
A site running external mode keeps its devices and check-in/out history in a
separate cmmc_usb database. There was no way to bring that in, so the choice
was to keep the second database forever or lose the history. This adds
'flask usb import-external' - dry run by default, --commit to write - which
reads through the same connection settings the routes already use and never
writes to the source.
Two things it has to reconcile, and they are why it is a command rather than a
few INSERTs.
The schemas disagree about SHAPE. External is an event log, one row per action;
self-hosted is a paired row carrying both times. Pairing walks each device in
time order, and real logs after years of use are untidy. The rule is that
nothing is invented and nothing vanishes: a drive still out becomes an open
checkout and is NOT flagged (that is just a drive somebody has), a return with
no matching issue is skipped and named, and two issues in a row leave the first
open rather than inventing a return time nobody recorded.
Identity is the other. usbdevices.serialnumber is unique and NOT NULL and the
external devices table has no serial, so device_id becomes it. Blank or
duplicated ids stop that device rather than inventing a key. external
device_owner has no column here, so it is appended to the device notes rather
than dropped.
Also adds usbcheckouts.scannedviruses, the one field with nowhere to land.
waswiped already means what external calls sanitized - selfhosted.py was
mapping it - but scanned_viruses was HARDCODED None in the log output: the API
promised the field and had nothing behind it, so an imported history would have
rendered blank. It now reads and writes properly.
Nullable, never defaulted to false. An unrecorded scan is not a failed scan,
and for removable-media records those are very different claims.
Full suite 2131 passed; seven new tests pin the pairing rules.
From the database review (verdict: sound-with-minor-issues). Applies the
actionable findings.
Redundant indexes: five non-unique secondary indexes duplicated a named idx_*
or a unique index on the same column - ix_communications_assetid,
ix_computers_hostname, ix_networkdevices_hostname, ix_printers_hostname (each
shadowing an idx_*), and idx_usb_serial (shadowing the serialnumber unique
index). Removed the redundant index source from the models (column index=True /
the extra db.Index) and added core migration 7d25 dropping the live duplicates.
The unique ix_*_assetid indexes are kept (they enforce assetid uniqueness).
Dead column: usbcheckouts.machineid was a NOT NULL soft-ref to the retired
machines table storing sentinel 0 (ADR-001). Dropped from the model + the
machineid=0 literal in selfhosted checkout; usb plugin migration 0002 drops it
live (downgrade restores it default 0).
Index: notifications.businessunitid (filtered by the shopfloor feed) was
unindexed; added index=True + notifications migration 0002.
CI: new migrations-mysql job proves the real multi-site deploy path - fresh
`flask db upgrade` + per-plugin install on utf8mb4 MySQL from empty, asserting
table count + charset and a clean second-run no-op. The pytest suite only
exercises SQLite create_all(), so a regression in the Alembic chain on MySQL
would otherwise ship undetected.
Verified: fresh core upgrade on a scratch utf8mb4 MySQL builds clean + no-op on
rerun (redundant indexes absent, unique assetid kept); plugin migrations applied
+ verified on the dev DB (machineid gone, bu index present). 953 backend tests
pass; naming + pyflakes green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Framework:
- Per-plugin Alembic migration chains (ADR-008): every bundled plugin
carries its own chain with a stamp-only anchor at the ownership cutover;
new plugin schema lands in plugins/<name>/migrations/, never the core
chain. Deploys add flask plugin upgrade-all. Fixed a latent bug in the
shared alembic template (engine URL resolution) and taught the metadata
filter to include FK-referenced core tables.
- Frontend plugin route gating (ADR-009): plugin routes carry meta.plugin;
a disabled plugin's pages redirect to the dashboard via a cached,
fail-open check against the new public GET /api/plugins/enabled.
- get_reports() plugin hook (contract 0.5.0 -> 0.6.0): plugins contribute
report cards; warranty and toner cards moved off the hardcoded list.
Reports:
- Hub grouped by category with search; inline reports render at the top,
are URL-backed (?report=id, back-button and deep links work), expose
their server-side filter params as controls, and export CSV. Warranty
and Toner pages gained CSV export.
- Deleted the dead legacy Warranty Status report (always-zero buckets
from a retired column).
Theming and fonts:
- Inter (variable) bundled locally via @fontsource, replacing the Google
Fonts Roboto import - air-gapped installs now render correctly; tables
use tabular numerals.
- Optional brand_primary_dark_color, brand_accent_color,
brand_sidebar_color settings applied to CSS vars at bootstrap.
USB frontend repair (views were reading a dead legacy shape):
- List/detail/form and the employee profile USB panels remapped to the
real API shape (device_id/device_desc/checkinoutlog); employee panels
now use /usb/checkouts endpoints; external-mode /usb/checkouts/active
honors the badge filter; dead client methods pruned.
Also: warranties list page no longer requires login (matches app
convention); collector doc rewritten with a GE-Enforce integration guide
and paste-ready PowerShell reporter; ADR index and CHANGELOG updated.
Verified: 323 tests pass, naming/style green, frontend builds, plugin
migration dry-run green on scratch MySQL.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The core chain already owns and reproduces the full bundled schema (deploys run
`flask db upgrade` only). The per-plugin Alembic baselines duplicated those
tables, so `flask plugin upgrade-all` would conflict - a footgun. Remove the 6
bundled plugin migration dirs; the per-plugin Alembic helpers
(alembic_template, PluginMigrationRunner) remain for external/filesystem
plugins. upgrade-all now cleanly no-ops for bundled plugins.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Each of the six bundled plugins (computers, equipment, network,
notifications, printers, usb) now has its own Alembic chain with a
baseline migration. Sister sites adopting one of these plugins can
manage its schema via `flask plugin migrate <name>` instead of relying
on db.create_all to bootstrap everything.
Existing single-site deploys that bootstrap via db.create_all continue
to work unchanged. The chains coexist; the bootstrap path stays the
operator's choice.
Framework
- shopdb/plugins/alembic_template.py: shared env.py logic + helpers.
PLUGIN_TABLE_OWNERS pins which tables belong to which plugin (explicit
registry, not import-side-effect). _get_plugin_metadata filters
db.metadata to only the named plugin's tables. create_plugin_tables /
drop_plugin_tables emit DDL via SQLAlchemy CreateTable so the table
definitions stay sourced from the models, not duplicated.
- shopdb/plugins/__init__.py: PluginManager.upgrade_all_plugins() runs
pending migrations across every discovered plugin and returns a status
dict. Idempotent (Alembic skips applied revisions).
CLI
- `flask plugin upgrade-all` runs pending migrations for every plugin.
Used on a fresh deploy after the core schema is in place.
Per-plugin scaffolding
- plugins/{computers,equipment,network,notifications,printers,usb}/
migrations/{alembic.ini, env.py, script.py.mako, versions/0001_baseline.py}
- Each env.py is a 5-line shim that sets PLUGIN_NAME and delegates to
the shared template. Each 0001_baseline calls create_plugin_tables(name)
/ drop_plugin_tables(name); no duplication of column definitions.
Tests
- tests/test_plugin_migrations.py (18 cases): every bundled plugin has
an entry in PLUGIN_TABLE_OWNERS, has the on-disk Alembic scaffolding,
and the filtered MetaData contains every owned table (catches drift
between the template's table list and what the models declare).
- 129 tests pass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
