Make a partly-undone import retryable, and refuse a plan it cannot deliver
Some checks failed
CI / naming (push) Successful in 3s
CI / frontend (push) Successful in 12s
CI / migrations-mysql (push) Successful in 41s
CI / backend (push) Failing after 4m44s

Five findings from the review of the WINC import, plus the rendering each fix
needed to be usable.

An undo that hit a conflict wrote status 'undone' anyway, and undo_batch returns
immediately for an undone batch. So the one entry that needed a retry was the
only one that could never get it: the human resolves the conflict, clicks Undo,
and gets a no-op. A conflicted undo is now 'partially-undone', undoneat is
stamped only when the batch is genuinely finished, and the batches table offers
"Retry undo" for that status. The column was String(16) and the new value is
exactly 16 characters, so migration 7d38 widens it to 32 rather than rely on an
exact fit.

_UNDO_REFERENCES listed only assets, so a business unit this batch created was
hard-deleted even when something else still pointed at it. Two of the three
missing referrers are real FKs (dashboarddefaults.businessunitid,
businessunits.parentid): MySQL raises and the whole undo rolls back, so the
batch cannot be undone at all. The third, notifications.businessunitid, has no
FK, so the delete succeeded and left a notification pointing at an id that no
longer exists - silently, on both engines. All four are listed and the guard is
tested through the route.

The reference guard also queried tables that need not exist. A lean site drops
the tables of plugins it does not install (ADR-014), and notifications is
plugin-owned, so on such a site the query raised 1146, rolled the undo back and
left the batch permanently un-undoable. Absent tables are skipped; a table that
does not exist cannot hold a reference.

_PACKET_GROUPS reports on location, but apply_plan only ever creates business
units - it sets locationid only when the location already resolved. A plan
needing a new location asked the admin to acknowledge it, got the tick, wrote
nothing, and reported nothing. build_packet now takes the groups the writer can
actually create; anything else needed is 'uncreatable' and blocks at plan time,
where the admin can act on it, rather than dying at commit on a message about
acknowledging something the screen had no control for. The writer refuses the
same case independently. Both the console packet and the import page name the
blocking groups: setting blocking without printing the reason would leave the
banner pointing at nothing.

The web undo route caught ValueError only, so refusing a batch belonging to
another importer surfaced as a 500. That refusal is new too: the journal is a
general primitive and undo_batch does not filter on kind, so any other
importer's batches were reversible through this endpoint, carrying WINC's
references map - which lists none of that importer's tables, making every
reference check pass and every delete unconditional.

Nine tests. The two guards were verified by planting violations: dropping
dashboarddefaults from the references map, and passing creatable=None, each fail
the test written for them.

Scope: product
This commit is contained in:
cproudlock
2026-09-03 09:09:55 -04:00
parent 405a2ef919
commit 377cc69264
12 changed files with 408 additions and 21 deletions

View File

@@ -57,7 +57,7 @@ def plan_hash(plan):
return hasher.hexdigest()
def build_packet(kind, plan, counts, groups):
def build_packet(kind, plan, counts, groups, creatable=None):
"""Build an approval packet from a plan.
kind a short label identifying the importer, stored on the packet.
@@ -69,6 +69,12 @@ def build_packet(kind, plan, counts, groups):
column is what a human would act on - the raw export value for
UNRESOLVED (so they know what to alias) and the canonical name for
NEEDS_CREATE (so they know what would be created).
creatable
the group names the caller's writer can actually create, or None
for "no opinion". A needs-create group outside this set is reported
under `uncreatable` and makes the packet blocking, because a plan
that promises rows nothing will create is as unrunnable as one with
unresolved values.
needscreate is keyed on the canonical name column when present; callers pass
the *_name column for that via the fourth tuple element when it differs from
@@ -86,6 +92,26 @@ def build_packet(kind, plan, counts, groups):
if needs_hits:
needscreate[group_name] = needs_hits
# A needs-create group the writer cannot actually create BLOCKS the plan,
# the same as an unresolved value. It is the same failure either way: the
# import cannot do what the preview says it would.
#
# It has to be decided HERE rather than at write time. The writer's own
# refusal sits behind the acknowledgement check, and the UI only ever
# acknowledges the groups it renders a control for - so a plan needing an
# uncreatable group reached Commit looking clean, and died on a message
# about acknowledging something the screen had no checkbox for. Blocking at
# plan time puts it in the preview, where the admin can act on it.
#
# creatable=None means "no opinion" - the CLI callers that do not pass it
# keep the old behaviour.
uncreatable = OrderedDict()
if creatable is not None:
allowed = set(creatable)
for group_name, hits in needscreate.items():
if group_name not in allowed:
uncreatable[group_name] = hits
return OrderedDict([
('kind', kind),
('batchid', str(uuid.uuid4())),
@@ -93,8 +119,9 @@ def build_packet(kind, plan, counts, groups):
('rowcount', len(plan)),
('counts', OrderedDict(counts)),
('needscreate', needscreate),
('uncreatable', uncreatable),
('unresolved', unresolved),
('blocking', bool(unresolved)),
('blocking', bool(unresolved) or bool(uncreatable)),
])
@@ -123,6 +150,20 @@ def format_packet(packet):
lines.append(' + %-28s (%d rows)' % (name, count))
lines.append(' total new %s rows: %d' % (group, total))
# An uncreatable group blocks, so it has to be ON SCREEN. Setting
# blocking without printing the reason left "BLOCKED (resolve the values
# above)" pointing at nothing.
uncreatable = packet.get('uncreatable') or {}
if uncreatable:
for group, items in uncreatable.items():
lines.append('')
lines.append(' BLOCKING: this import cannot create %s rows, but '
'%d are needed - create them first, or alias the '
'export values to rows that exist:'
% (group, len(items)))
for name, count in items.items():
lines.append(' x %-28s (%d rows)' % (name, count))
unresolved = packet.get('unresolved') or {}
if unresolved:
for group, items in unresolved.items():
@@ -134,7 +175,7 @@ def format_packet(packet):
lines.append('')
lines.append(' status : %s' % (
'BLOCKED (resolve the values above)' if packet.get('blocking')
'BLOCKED (resolve the problems above)' if packet.get('blocking')
else 'READY for acknowledgment'))
return '\n'.join(lines)