A site running external mode keeps its devices and check-in/out history in a
separate cmmc_usb database. There was no way to bring that in, so the choice
was to keep the second database forever or lose the history. This adds
'flask usb import-external' - dry run by default, --commit to write - which
reads through the same connection settings the routes already use and never
writes to the source.
Two things it has to reconcile, and they are why it is a command rather than a
few INSERTs.
The schemas disagree about SHAPE. External is an event log, one row per action;
self-hosted is a paired row carrying both times. Pairing walks each device in
time order, and real logs after years of use are untidy. The rule is that
nothing is invented and nothing vanishes: a drive still out becomes an open
checkout and is NOT flagged (that is just a drive somebody has), a return with
no matching issue is skipped and named, and two issues in a row leave the first
open rather than inventing a return time nobody recorded.
Identity is the other. usbdevices.serialnumber is unique and NOT NULL and the
external devices table has no serial, so device_id becomes it. Blank or
duplicated ids stop that device rather than inventing a key. external
device_owner has no column here, so it is appended to the device notes rather
than dropped.
Also adds usbcheckouts.scannedviruses, the one field with nowhere to land.
waswiped already means what external calls sanitized - selfhosted.py was
mapping it - but scanned_viruses was HARDCODED None in the log output: the API
promised the field and had nothing behind it, so an imported history would have
rendered blank. It now reads and writes properly.
Nullable, never defaulted to false. An unrecorded scan is not a failed scan,
and for removable-media records those are very different claims.
Full suite 2131 passed; seven new tests pin the pairing rules.
From the database review (verdict: sound-with-minor-issues). Applies the
actionable findings.
Redundant indexes: five non-unique secondary indexes duplicated a named idx_*
or a unique index on the same column - ix_communications_assetid,
ix_computers_hostname, ix_networkdevices_hostname, ix_printers_hostname (each
shadowing an idx_*), and idx_usb_serial (shadowing the serialnumber unique
index). Removed the redundant index source from the models (column index=True /
the extra db.Index) and added core migration 7d25 dropping the live duplicates.
The unique ix_*_assetid indexes are kept (they enforce assetid uniqueness).
Dead column: usbcheckouts.machineid was a NOT NULL soft-ref to the retired
machines table storing sentinel 0 (ADR-001). Dropped from the model + the
machineid=0 literal in selfhosted checkout; usb plugin migration 0002 drops it
live (downgrade restores it default 0).
Index: notifications.businessunitid (filtered by the shopfloor feed) was
unindexed; added index=True + notifications migration 0002.
CI: new migrations-mysql job proves the real multi-site deploy path - fresh
`flask db upgrade` + per-plugin install on utf8mb4 MySQL from empty, asserting
table count + charset and a clean second-run no-op. The pytest suite only
exercises SQLite create_all(), so a regression in the Alembic chain on MySQL
would otherwise ship undetected.
Verified: fresh core upgrade on a scratch utf8mb4 MySQL builds clean + no-op on
rerun (redundant indexes absent, unique assetid kept); plugin migrations applied
+ verified on the dev DB (machineid gone, bu index present). 953 backend tests
pass; naming + pyflakes green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Make the app distributable to other GE Aerospace sites (one self-hosted
instance per site, ADR-004). GE values remain the shipped defaults; every
site-specific behavior is now a Setting an admin can change in the UI.
Settings-driven site config:
- Branding: site/QR/badge logos, favicon, primary color (upload endpoints
mirror the map-blueprint pattern; new Settings > Branding section).
- ServiceNow: search/incident/change URL templates ({ticket}), ticket
prefixes, enable toggle. Defaults point at the current
geaerospaceqa.service-now.com global search. Disabled = plain-text tickets.
- Employee-id regex (employeeid_pattern), printer hostname template,
QR label targets (qr_target_printer / qr_target_usb, blank = asset page,
else URL template with placeholders), usb_label_style (barcode|qr).
- West Jefferson floor-plan PNGs removed from the tree; generic placeholder
ships as the map default and sites upload their own blueprint.
Security closeout:
- dashboarddefaults writes now require admin.
- Collector: generic error messages (no str(exc) leak); API key accepted
via X-API-Key header only (BREAKING: querystring api_key removed).
- IP-based login rate limiting (AUTH_RATELIMIT_* knobs) atop account lockout.
- Setting.set() creation race fixed (IntegrityError retry).
Release engineering and docs:
- __version__ 0.5.0 (distinct from __contract_version__, ADR-007),
CHANGELOG.md, Gitea Actions CI config, frontend version aligned.
- One wizard-first install story across README/DEPLOY; new CONFIG.md,
UPGRADE.md, BACKUP-RESTORE.md; CLAUDE.md and ROADMAP de-staled.
- Dockerfile multi-stage build now bundles the frontend; compose binds
MySQL to 127.0.0.1; stale database/schema.sql and one-off SQL removed.
Debt and fixes:
- .query.get() -> db.session.get() sweep; datetime.utcnow() removed
(naive-UTC via timezone-aware now); users.py on authz decorators.
- Fixed 4 stale tests (slides feed shape, shopfloor splitperemployee,
plugin contract purity) and the USB label page field mapping (both usb
modes emit the cmmc shape: device_id/device_desc).
- Health endpoint reports the real version.
248 tests pass; naming/style check green; frontend builds; fresh-DB
flask db upgrade + seeds verified; QR targets verified by decoding
rendered codes.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Plugins were reaching into internal core paths (shopdb.core.models.*,
shopdb.extensions, shopdb.utils.*), coupling them to core's file layout and
violating the ADR-001 contract. Consolidate onto one versioned surface.
- shopdb.api: expand from 2 helpers to the full plugin import surface -
db, cache; BaseModel, AuditMixin; core models (Asset, AssetType,
AssetStatus, Vendor, Model, Communication, CommunicationType, Location,
Setting, AuditLog, Application, AppVersion, OperatingSystem); response +
pagination helpers; employee_connection. Documented in PLUGIN-HOOKS.md.
- Migrate all 22 plugin source files to import only from shopdb.api (plus
shopdb.plugins.base for the ABC).
- Drop the printers plugin's legacy MachineType dependency: remove
_ensure_legacy_machine_types and the seed_supplies machinetypeid lookup
(Model.machinetypeid is nullable; printers carry type via PrinterType).
- Guard test test_plugins_only_import_contract_surface scans plugin source
and fails on any core import outside shopdb.api / shopdb.plugins.base.
- Scaffold templates updated so generated plugins are contract-pure.
- Bump __contract_version__ 0.2.0 -> 0.3.0 (additive surface expansion;
manifests pin <1.0.0 so they still satisfy).
145 tests pass, naming/style green, app factory boots all 6 plugins.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Fix equipment badge barcode not rendering (loading race condition)
- Fix printer QR code not rendering on initial load (same race condition)
- Add model image to equipment badge via imageurl from Model table
- Fix white-on-white machine number text on badge, tighten barcode spacing
- Add PaginationBar component used across all list pages
- Split monolithic router into per-plugin route modules
- Fix 25 GET API endpoints returning 401 (jwt_required -> optional=True)
- Align list page columns across Equipment, PCs, and Network pages
- Add print views: EquipmentBadge, PrinterQRSingle, PrinterQRBatch, USBLabelBatch
- Add PC Relationships report, migration docs, and CLAUDE.md project guide
- Various plugin model, API, and frontend refinements
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add equipmentApi and computersApi to replace legacy machinesApi
- Add controller vendor/model fields to Equipment model and forms
- Fix map marker navigation to use plugin-specific IDs (equipmentid,
computerid, printerid, networkdeviceid) instead of assetid
- Fix search to use unified Asset table with correct plugin IDs
- Remove legacy printer search that used non-existent field names
- Enable optional JWT auth for detail endpoints (public read access)
- Clean up USB plugin models (remove unused checkout model)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New Plugins:
- USB plugin: Device checkout/checkin with employee lookup, checkout history
- Notifications plugin: Announcements with types, scheduling, shopfloor display
- Network plugin: Network device management with subnets and VLANs
- Equipment and Computers plugins: Asset type separation
Frontend:
- EmployeeSearch component: Reusable employee lookup with autocomplete
- USB views: List, detail, checkout/checkin modals
- Notifications views: List, form with recognition mode
- Network views: Device list, detail, form
- Calendar view with FullCalendar integration
- Shopfloor and TV dashboard views
- Reports index page
- Map editor for asset positioning
- Light/dark mode fixes for map tooltips
Backend:
- Employee search API with external lookup service
- Collector API for PowerShell data collection
- Reports API endpoints
- Slides API for TV dashboard
- Fixed AppVersion model (removed BaseModel inheritance)
- Added checkout_name column to usbcheckouts table
Styling:
- Unified detail page styles
- Improved pagination (page numbers instead of prev/next)
- Dark/light mode theme improvements
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
