cbdfcbabb774a1d84aff97bc0c3f62d2622e05fb
9 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
cproudlock
|
cbdfcbabb7 |
One badge vocabulary, and images a browser can actually load
Two operator-visible defects and three doc corrections, closing the tail of the 2026-09-03 run. THE BADGE. The two kiosks read the same physical badges and disagreed. The parts kiosk understood both shapes a scanner emits - the bare SSO and the PayNo wrapper 0<digits>BZ - while the USB kiosk compared the raw string to an SSO, so the wrapper resolved to nobody. An operator whose keyboard-wedge scanner emits it could take a 3D-printed part and could not check out a drive, with the same badge, at two cabinets a few metres apart. The shape now lives in shopdb/utils/badges.py on the contract surface, because a plugin may not import another plugin and because a badge is a property of the site rather than of a cabinet. Both kiosks use it, and both store the DIGITS rather than the string that arrived: one row per spelling makes a custody history that cannot be joined to itself. That closed a subtler hole in the same place. str.isdigit() is true for Arabic-Indic and other decimal forms, and int() parses them, so a badge written in those characters was accepted, matched a REAL employee, and was then stored as an SSO string nothing else matches. ASCII digits only. An empty scan and an unreadable one now get different answers, because they are different problems for the person at the cabinet. THE IMAGES. /api/applications/image/<filename> and /api/models/image/<filename> required a permission, and a browser fetching <img src="..."> sends no Authorization header - so the gate did not protect the file, it guaranteed the picture never appeared. Both are ungated now, like the four sibling image routes (branding logo, map blueprint, employee photo, part picture), and registered in PUBLIC_ENDPOINTS with that reasoning. /api/printers/lookup?fqdn= never matched anything: both arguments were compared against Communication.ipaddress, so a hostname lookup 404d on printers whose hostname was on the record. An FQDN now matches the hostname, with a fallback to the short name. The slide manager gained the per-slide hold time. The PATCH route and the feed have always carried `seconds`; nothing called it, so the one documented per-slide control was curl-only. Docs: API-REFERENCE listed the employee lookups and the whole notifications surface as unauthenticated when they answer 401, and called both kiosks "deliberately open" when they are off by default and refuse anything without the site's device token. The installer download says applications.download, which is what the code requires. api-inventory.json said printers/lookup was jwt-optional. Fourteen tests. The badge rule verified by planting the old per-kiosk behaviour: three fail, one in each kiosk. Scope: product |
||
|
cproudlock
|
aa8028f96e |
Let the row refuse the write both kiosks were deciding in Python
Wave 1 of the remediation plan for the 2026-09-03 feature test run. Two kiosks, one shape of defect, one critical consequence. Both kiosks read the state, decided in Python, and then wrote. Nothing stops two requests interleaving between the read and the write, so both see the same pre-state and both proceed. Reproduced on MySQL 5.6 over HTTP, five concurrent takes of 5 from a bin holding 5: all five were accepted and quantityonhand landed on -20, because each request wrote its own arithmetic. The ledger had recorded every one of them, so the cached quantity and the ledger disagreed permanently, which is the number the shop floor actually reads. The USB kiosk had the same shape: concurrent scans of one drive all succeeded and inserted several open custody rows, so one drive was recorded out to two people and the documented 409 never fired. The comparison now lives in the WHERE clause, where the database settles it at write time. printedparts._move_quantity issues UPDATE ... WHERE quantityonhand >= :n and treats rowcount 0 as the refusal, raising InsufficientStock, which take and adjust turn back into the message they already gave - carrying the quantity the database holds now, not the stale one the request read. The low-stock alert derives its before-value from what the write produced for the same reason. Both USB paths claim the device with ischeckedout (self-hosted) or status (external cmmc_usb) in the WHERE clause; the external one claims BEFORE writing the log line, so a refused claim leaves nothing to undo. Check-in is guarded the same way. After the fix, the same five concurrent takes: one accepted, four refused, quantityonhand 0 and the ledger sum 0. Check-in also closes EVERY open row for the device rather than the newest. One device can only be out once, so a second open row is corruption, and closing only the newest left the older ones open forever - the device then read as returned while its custody history said otherwise. Separately, usbcheckouts now records who handed a drive back (checkinsso/checkinname, migration usb0004checkinidentity). Returning for a colleague is a supported flow that the kiosk asks to confirm, but the row it wrote attributed the check-in to the HOLDER, so the custody history and the audit report both named somebody who was not there. Nullable and deliberately not backfilled: copying the holder's badge into old rows would manufacture evidence for the exact claim this stops making. Four tests. Both guards verified by planting: strip the condition out of either UPDATE and its test fails. Scope: product |
||
|
cproudlock
|
60c35728e8 |
Import USB records off an external cmmc_usb database, so a site can retire it
A site running external mode keeps its devices and check-in/out history in a separate cmmc_usb database. There was no way to bring that in, so the choice was to keep the second database forever or lose the history. This adds 'flask usb import-external' - dry run by default, --commit to write - which reads through the same connection settings the routes already use and never writes to the source. Two things it has to reconcile, and they are why it is a command rather than a few INSERTs. The schemas disagree about SHAPE. External is an event log, one row per action; self-hosted is a paired row carrying both times. Pairing walks each device in time order, and real logs after years of use are untidy. The rule is that nothing is invented and nothing vanishes: a drive still out becomes an open checkout and is NOT flagged (that is just a drive somebody has), a return with no matching issue is skipped and named, and two issues in a row leave the first open rather than inventing a return time nobody recorded. Identity is the other. usbdevices.serialnumber is unique and NOT NULL and the external devices table has no serial, so device_id becomes it. Blank or duplicated ids stop that device rather than inventing a key. external device_owner has no column here, so it is appended to the device notes rather than dropped. Also adds usbcheckouts.scannedviruses, the one field with nowhere to land. waswiped already means what external calls sanitized - selfhosted.py was mapping it - but scanned_viruses was HARDCODED None in the log output: the API promised the field and had nothing behind it, so an imported history would have rendered blank. It now reads and writes properly. Nullable, never defaulted to false. An unrecorded scan is not a failed scan, and for removable-media records those are very different claims. Full suite 2131 passed; seven new tests pin the pairing rules. |
||
|
cproudlock
|
f59aca5a1b |
USB check-in / check-out audit, as a report rather than a paged list
CMMC removable-media tracking is an audit obligation, and answering "who had this drive, and when" meant paging through the check-in/out list by hand. GET /api/usb/reports/audit returns the events - a take and a return are separate rows, because a count cannot answer the question an audit actually asks - with summary figures riding along for the people who only want the shape of it, and a CSV for the people who want it in a spreadsheet. The window is named rather than a day count: today, last month, six months, a year, all time. Nobody should have to work out that a year is 365. Both directory modes answer in one voice. The external path reads the cmmc_usb checkinoutlog; self-hosted expands one usbcheckouts row into its two ends, so a returned drive is distinguishable from one still out, and the window filter runs in SQL against either end of the loan - a drive taken before the window and returned inside it is a check-in the window has to show. Rows come back from the mode-specific half; the CSV, the summary and the response shape are built once in the route. Ordering is total in both modes. Timestamp alone ties for two events in the same second, and the same query then returns them in a different order each run, which reads as the log being unstable. |
||
|
cproudlock
|
7c47ae1f7c |
USB kiosk: take and return a drive without a ShopDB login
The people who actually take USB drives are standing at the cabinet, and most of them have no account here. Check-out and check-in were gated behind a login that was not doing much anyway: both modes already attribute the action to a scanned BADGE rather than to the signed-in user, so the login was gating the screen, not the record. /usb-kiosk is a full-screen touch page modelled on the parts kiosk, off unless a site sets usb_kiosk_enabled - every route 404s until then, rather than 403, so a disabled kiosk does not advertise that there is a door here. It works against the self-hosted tables or an existing cmmc_usb database. THE LIST IS THE TRADE. These drives carry no barcode, so unlike the parts kiosk there is nothing to present and the screen has to offer a list - which exposes more than a scan-first design, where you had to already hold a drive to learn anything. So the list is not anonymous: an SSO comes first and must resolve to a real person before any drive is named, which is the same bar taking a drive already had to clear. It is a POST so the badge stays out of proxy access logs. The main screen shows the caller's own drives, what is free, and a COUNT of everything else; holders of other people's drives appear only behind a second deliberate tap, so nobody reads the whole cabinet off a wall panel in passing. A badge that resolves to no employee is REFUSED, unlike the signed-in route which creates the user row. A checkout nobody can trace to a person defeats the tracking, and at a kiosk there is no operator standing there to vouch for it. Returning somebody else's drive asks first and then records the badge that was SCANNED, so the log says who actually handed it back. That is the honest record and it needs no change to a schema this app does not own. Refusing a mismatch outright is a setting but not the default: a drive left behind by somebody on leave would otherwise be unreturnable at the kiosk, which just moves the problem to a person with a login. The authz sweep caught these writes, twice - once for the two mutations and again for the two list endpoints. Both are exempted with the reasoning written out beside the parts-kiosk entry rather than silenced. The list endpoints are POSTs that change nothing. Verified by driving the real page: SSO on the keypad, tap Take, database moves. |
||
|
cproudlock
|
1c6c7ba14b |
DB review fixes: drop redundant indexes + dead column, add CI MySQL-upgrade job
From the database review (verdict: sound-with-minor-issues). Applies the actionable findings. Redundant indexes: five non-unique secondary indexes duplicated a named idx_* or a unique index on the same column - ix_communications_assetid, ix_computers_hostname, ix_networkdevices_hostname, ix_printers_hostname (each shadowing an idx_*), and idx_usb_serial (shadowing the serialnumber unique index). Removed the redundant index source from the models (column index=True / the extra db.Index) and added core migration 7d25 dropping the live duplicates. The unique ix_*_assetid indexes are kept (they enforce assetid uniqueness). Dead column: usbcheckouts.machineid was a NOT NULL soft-ref to the retired machines table storing sentinel 0 (ADR-001). Dropped from the model + the machineid=0 literal in selfhosted checkout; usb plugin migration 0002 drops it live (downgrade restores it default 0). Index: notifications.businessunitid (filtered by the shopfloor feed) was unindexed; added index=True + notifications migration 0002. CI: new migrations-mysql job proves the real multi-site deploy path - fresh `flask db upgrade` + per-plugin install on utf8mb4 MySQL from empty, asserting table count + charset and a clean second-run no-op. The pytest suite only exercises SQLite create_all(), so a regression in the Alembic chain on MySQL would otherwise ship undetected. Verified: fresh core upgrade on a scratch utf8mb4 MySQL builds clean + no-op on rerun (redundant indexes absent, unique assetid kept); plugin migrations applied + verified on the dev DB (machineid gone, bu index present). 953 backend tests pass; naming + pyflakes green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
cproudlock
|
46e50c07ff |
Add the API import surface for legacy migrations (contract 0.8.0)
Goal: an LLM or script can migrate an entire legacy database using only the HTTP API - original history preserved, safely re-runnable. - X-Import-Mode header (admin only): create/update endpoints across 15 timestamped entity types accept original createddate/modifieddate; helper exposed via shopdb.api (contract 0.7.0 -> 0.8.0). - Exact-match natural-key lookup filters on 13 list endpoints for the lookup-then-upsert recipe. - Selfhosted USB checkout/checkin accept backdated event times in import mode. - docs/IMPORT-API.md: operator manual grounded in the real legacy schema - order of operations, full table-by-table mapping including the machines fan-out, idempotent Python importer with dry-run, parity checks, and decided dispositions for unmigrated tables (DNC config stays live-fed via the collector; supportteams/appowners map to the upcoming supportteams model). 635 tests pass; naming green; frontend untouched. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
cproudlock
|
b8c22244a1 |
Multi-site distribution readiness: settings-driven site config, security closeout, release engineering, v0.5.0
Make the app distributable to other GE Aerospace sites (one self-hosted
instance per site, ADR-004). GE values remain the shipped defaults; every
site-specific behavior is now a Setting an admin can change in the UI.
Settings-driven site config:
- Branding: site/QR/badge logos, favicon, primary color (upload endpoints
mirror the map-blueprint pattern; new Settings > Branding section).
- ServiceNow: search/incident/change URL templates ({ticket}), ticket
prefixes, enable toggle. Defaults point at the current
geaerospaceqa.service-now.com global search. Disabled = plain-text tickets.
- Employee-id regex (employeeid_pattern), printer hostname template,
QR label targets (qr_target_printer / qr_target_usb, blank = asset page,
else URL template with placeholders), usb_label_style (barcode|qr).
- West Jefferson floor-plan PNGs removed from the tree; generic placeholder
ships as the map default and sites upload their own blueprint.
Security closeout:
- dashboarddefaults writes now require admin.
- Collector: generic error messages (no str(exc) leak); API key accepted
via X-API-Key header only (BREAKING: querystring api_key removed).
- IP-based login rate limiting (AUTH_RATELIMIT_* knobs) atop account lockout.
- Setting.set() creation race fixed (IntegrityError retry).
Release engineering and docs:
- __version__ 0.5.0 (distinct from __contract_version__, ADR-007),
CHANGELOG.md, Gitea Actions CI config, frontend version aligned.
- One wizard-first install story across README/DEPLOY; new CONFIG.md,
UPGRADE.md, BACKUP-RESTORE.md; CLAUDE.md and ROADMAP de-staled.
- Dockerfile multi-stage build now bundles the frontend; compose binds
MySQL to 127.0.0.1; stale database/schema.sql and one-off SQL removed.
Debt and fixes:
- .query.get() -> db.session.get() sweep; datetime.utcnow() removed
(naive-UTC via timezone-aware now); users.py on authz decorators.
- Fixed 4 stale tests (slides feed shape, shopfloor splitperemployee,
plugin contract purity) and the USB label page field mapping (both usb
modes emit the cmmc shape: device_id/device_desc).
- Health endpoint reports the real version.
248 tests pass; naming/style check green; frontend builds; fresh-DB
flask db upgrade + seeds verified; QR targets verified by decoding
rendered codes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
cproudlock
|
8c198938c2 |
Complete USB self-hosted mode (app tables, no external cmmc_usb)
"Create the tables here" for USB now actually works. Add selfhosted.py backing every USB endpoint with the app-owned usbdevices/usbcheckouts tables and returning the same response shape as the external cmmc_usb path. Each route in routes.py delegates to it when usb_directory_mode=selfhosted (the default). - Device status derives from ischeckedout + isactive (retired); check-in/out history is synthesized from usbcheckouts rows (check-out + check-in events); holder names resolve from the self-hosted employee directory. - External cmmc_usb mode unchanged, for sites already running that solution. Verified E2E: create -> list -> checkout -> checkin -> history all correct against the app tables. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |