Let the row refuse the write both kiosks were deciding in Python

Wave 1 of the remediation plan for the 2026-09-03 feature test run. Two kiosks,
one shape of defect, one critical consequence.

Both kiosks read the state, decided in Python, and then wrote. Nothing stops two
requests interleaving between the read and the write, so both see the same
pre-state and both proceed. Reproduced on MySQL 5.6 over HTTP, five concurrent
takes of 5 from a bin holding 5: all five were accepted and quantityonhand
landed on -20, because each request wrote its own arithmetic. The ledger had
recorded every one of them, so the cached quantity and the ledger disagreed
permanently, which is the number the shop floor actually reads. The USB kiosk
had the same shape: concurrent scans of one drive all succeeded and inserted
several open custody rows, so one drive was recorded out to two people and the
documented 409 never fired.

The comparison now lives in the WHERE clause, where the database settles it at
write time. printedparts._move_quantity issues
UPDATE ... WHERE quantityonhand >= :n and treats rowcount 0 as the refusal,
raising InsufficientStock, which take and adjust turn back into the message they
already gave - carrying the quantity the database holds now, not the stale one
the request read. The low-stock alert derives its before-value from what the
write produced for the same reason. Both USB paths claim the device with
ischeckedout (self-hosted) or status (external cmmc_usb) in the WHERE clause;
the external one claims BEFORE writing the log line, so a refused claim leaves
nothing to undo. Check-in is guarded the same way.

After the fix, the same five concurrent takes: one accepted, four refused,
quantityonhand 0 and the ledger sum 0.

Check-in also closes EVERY open row for the device rather than the newest.
One device can only be out once, so a second open row is corruption, and
closing only the newest left the older ones open forever - the device then read
as returned while its custody history said otherwise.

Separately, usbcheckouts now records who handed a drive back
(checkinsso/checkinname, migration usb0004checkinidentity). Returning for a
colleague is a supported flow that the kiosk asks to confirm, but the row it
wrote attributed the check-in to the HOLDER, so the custody history and the
audit report both named somebody who was not there. Nullable and deliberately
not backfilled: copying the holder's badge into old rows would manufacture
evidence for the exact claim this stops making.

Four tests. Both guards verified by planting: strip the condition out of either
UPDATE and its test fails.

Scope: product
This commit is contained in:
cproudlock
2026-09-03 13:58:26 -04:00
parent 377cc69264
commit aa8028f96e
8 changed files with 362 additions and 23 deletions

View File

@@ -72,8 +72,9 @@ EXPECTED_HEAD_REVISION['machines'] = 'machines0002rename'
EXPECTED_HEAD_REVISION['employees'] = 'employees0002photo'
# usb drops the dead usbcheckouts.machineid column on top of its anchor, then
# adds scannedviruses so an imported external history keeps the scan result -
# the field a removable-media audit actually asks about.
EXPECTED_HEAD_REVISION['usb'] = 'usb0003scanfields'
# the field a removable-media audit actually asks about - and finally records
# who handed a drive back, which is not always who took it out.
EXPECTED_HEAD_REVISION['usb'] = 'usb0004checkinidentity'
# network links a device to a catalog model, which is where its photo comes
# from - machines, PCs and printers already had that link.
EXPECTED_HEAD_REVISION['network'] = 'network0003prefix'

View File

@@ -532,3 +532,65 @@ def test_the_badge_route_is_not_an_enumeration_oracle(client, item, auth_headers
assert unknown.status_code == malformed.status_code == 422
assert (unknown.get_json()['data']['error']['message']
== malformed.get_json()['data']['error']['message'])
# --- the race ---------------------------------------------------------------
#
# The take read quantityonhand, compared it in Python, and then wrote. Five
# concurrent takes of 5 from a bin holding 5 all passed that comparison and all
# wrote their own 5 - 5 = 0, so quantityonhand ended up permanently out of step
# with the ledger that had recorded every one of them. The comparison now lives
# in the UPDATE's WHERE clause, where it is atomic.
def test_a_take_that_passed_its_precheck_is_still_refused_by_the_row(
client, auth_headers, app, item, directory_employee, kiosk,
monkeypatch):
"""Simulate the race window: the request holds an item that says 5 on hand
while the row has already been emptied by somebody else."""
from plugins.printedparts.api import routes
from plugins.printedparts.models import PrintedItem
stocked = client.post(f'/api/printedparts/items/{item}/restock',
json={'quantity': 5, 'badge': directory_employee},
headers=auth_headers)
assert stocked.status_code == 200
real_find = routes._kiosk_find_item
def stale_find(itemcode):
"""What the losing request sees: its own read, taken before the winner
committed, still shows a full bin.
Detached first, so setting the attribute changes only this request's
view and never reaches the database - the test is about a stale
reader, not about a second writer.
"""
found = real_find(itemcode)
if found is not None:
db.session.expunge(found)
found.quantityonhand = 5
return found
# Somebody else empties the bin first, through the kiosk, so the ledger
# records it the way a real competing take would.
winner = client.post('/api/printedparts/kiosk/take',
json={'itemcode': '3DP-9001', 'quantity': 5,
'badge': directory_employee},
headers=KIOSK_HEADERS)
assert winner.status_code == 200, winner.get_json()
monkeypatch.setattr(routes, '_kiosk_find_item', stale_find)
taken = client.post('/api/printedparts/kiosk/take',
json={'itemcode': '3DP-9001', 'quantity': 5,
'badge': directory_employee},
headers=KIOSK_HEADERS)
assert taken.status_code == 400, taken.get_json()
assert 'on hand' in taken.get_json()['data']['error']['message']
with app.app_context():
after = db.session.query(PrintedItem).filter_by(
printeditemid=item).first()
assert after.quantityonhand == 0, 'stock must not go negative'
ledger = sum(t.quantitychange for t in after.transactions)
assert ledger == after.quantityonhand, \
'quantityonhand and the ledger must agree'

View File

@@ -314,3 +314,85 @@ def test_the_token_alone_is_not_enough_when_the_kiosk_is_off(client, kiosk):
_setting(kiosk, 'usb_kiosk_enabled', 'false')
assert client.get('/api/usb/kiosk/device/USB-1',
headers=KIOSK_HEADERS).status_code == 404
# --- the race, and who returned it ------------------------------------------
#
# The kiosk decided "is this drive available?" in Python and then wrote. Two
# scans of the same drive that interleaved both read "available", so both
# inserted a custody row: one drive recorded out to two people, and the
# documented 409 never fired. The fix puts ischeckedout in the WHERE clause, so
# the database settles it at write time.
def test_a_checkout_that_passed_its_availability_check_is_still_refused(
client, kiosk, monkeypatch):
"""Simulate the race window: the request holds a device object that says
available while the row has already been claimed by somebody else."""
from plugins.usb.api import selfhosted
from plugins.usb.models import USBDevice
first = _out(client, badge='270015376')
assert first.status_code == 200
# What the losing request sees: its own read, taken before the winner
# committed, still says the drive is on the shelf.
stale = USBDevice.query.filter_by(serialnumber='USB-1').first()
monkeypatch.setattr(
selfhosted, '_find',
lambda device_id: _stale_copy(stale))
second = _out(client, badge='270099999')
assert second.status_code == 409, second.get_json()
open_rows = _open_checkouts(USBDevice)
assert len(open_rows) == 1, 'one drive, one open custody row'
assert open_rows[0].sso == '270015376'
def _stale_copy(device):
"""A detached stand-in that still believes the drive is available."""
class Stale:
usbdeviceid = device.usbdeviceid
serialnumber = device.serialnumber
isactive = True
ischeckedout = False
currentuserid = None
currentusername = None
return Stale()
def _open_checkouts(_):
from plugins.usb.models import USBCheckout
return USBCheckout.query.filter_by(checkintime=None).all()
def test_a_return_records_who_handed_it_back_not_only_the_holder(client, kiosk):
"""Returning for a colleague is a supported flow, so the check-in has to
name the person who actually walked it back."""
from plugins.usb.models import USBCheckout
assert _out(client, badge='270015376').status_code == 200
returned = _in(client, badge='270099999', forsomeoneelse=True)
assert returned.status_code == 200, returned.get_json()
row = USBCheckout.query.order_by(USBCheckout.checkoutid.desc()).first()
assert row.sso == '270015376', 'the holder still owns the checkout'
assert row.checkinsso == '270099999', 'the returner is recorded'
assert row.checkinname and 'Sam' in row.checkinname
def test_a_check_in_closes_every_open_row_for_that_device(client, kiosk, db):
"""Defence in depth for history written before the guard existed: a device
can only be out once, so a second open row is corruption, and closing only
the newest left the older ones open forever."""
from plugins.usb.models import USBCheckout, USBDevice
assert _out(client, badge='270015376').status_code == 200
device = USBDevice.query.filter_by(serialnumber='USB-1').first()
db.session.add(USBCheckout(usbdeviceid=device.usbdeviceid, sso='270099999',
checkoutname='Sam Okafor'))
db.session.commit()
assert len(_open_checkouts(USBDevice)) == 2
assert _in(client, badge='270015376').status_code == 200
assert _open_checkouts(USBDevice) == []