Import USB records off an external cmmc_usb database, so a site can retire it

A site running external mode keeps its devices and check-in/out history in a
separate cmmc_usb database. There was no way to bring that in, so the choice
was to keep the second database forever or lose the history. This adds
'flask usb import-external' - dry run by default, --commit to write - which
reads through the same connection settings the routes already use and never
writes to the source.

Two things it has to reconcile, and they are why it is a command rather than a
few INSERTs.

The schemas disagree about SHAPE. External is an event log, one row per action;
self-hosted is a paired row carrying both times. Pairing walks each device in
time order, and real logs after years of use are untidy. The rule is that
nothing is invented and nothing vanishes: a drive still out becomes an open
checkout and is NOT flagged (that is just a drive somebody has), a return with
no matching issue is skipped and named, and two issues in a row leave the first
open rather than inventing a return time nobody recorded.

Identity is the other. usbdevices.serialnumber is unique and NOT NULL and the
external devices table has no serial, so device_id becomes it. Blank or
duplicated ids stop that device rather than inventing a key. external
device_owner has no column here, so it is appended to the device notes rather
than dropped.

Also adds usbcheckouts.scannedviruses, the one field with nowhere to land.
waswiped already means what external calls sanitized - selfhosted.py was
mapping it - but scanned_viruses was HARDCODED None in the log output: the API
promised the field and had nothing behind it, so an imported history would have
rendered blank. It now reads and writes properly.

Nullable, never defaulted to false. An unrecorded scan is not a failed scan,
and for removable-media records those are very different claims.

Full suite 2131 passed; seven new tests pin the pairing rules.
This commit is contained in:
cproudlock
2026-08-29 13:02:21 -04:00
parent 99c40ae7c3
commit 60c35728e8
7 changed files with 404 additions and 3 deletions

View File

@@ -0,0 +1,49 @@
"""Record whether a returned drive was virus-scanned.
The external cmmc_usb checkinoutlog carries scanned_viruses and sanitized on
every row. The self-hosted usbcheckouts table already has waswiped, which is
the same fact as sanitized under a different name, but nothing corresponds to
scanned_viruses. That gap was harmless while the two backends were
alternatives, and becomes data loss the moment a site imports its history and
cuts over: whether a returned drive was scanned is precisely what a
removable-media audit asks.
Nullable on purpose, and NOT defaulted to false. A checkout recorded before
this column existed is not a checkout where the drive went unscanned; it is one
where nobody wrote it down. False would assert something the data does not say.
Idempotent, so a re-run on an already-migrated database is a no-op.
Revision ID: usb0003scanfields
Revises: usb0002dropmachineid
"""
from alembic import op
import sqlalchemy as sa
revision = 'usb0003scanfields'
down_revision = 'usb0002dropmachineid'
branch_labels = None
depends_on = None
def upgrade():
bind = op.get_bind()
insp = sa.inspect(bind)
if 'usbcheckouts' not in insp.get_table_names():
return
cols = {c['name'] for c in insp.get_columns('usbcheckouts')}
if 'scannedviruses' not in cols:
op.add_column('usbcheckouts',
sa.Column('scannedviruses', sa.Boolean(), nullable=True))
def downgrade():
bind = op.get_bind()
insp = sa.inspect(bind)
if 'usbcheckouts' not in insp.get_table_names():
return
cols = {c['name'] for c in insp.get_columns('usbcheckouts')}
if 'scannedviruses' in cols:
op.drop_column('usbcheckouts', 'scannedviruses')