Make a partly-undone import retryable, and refuse a plan it cannot deliver

Five findings from the review of the WINC import, plus the rendering each fix
needed to be usable.

An undo that hit a conflict wrote status 'undone' anyway, and undo_batch returns
immediately for an undone batch. So the one entry that needed a retry was the
only one that could never get it: the human resolves the conflict, clicks Undo,
and gets a no-op. A conflicted undo is now 'partially-undone', undoneat is
stamped only when the batch is genuinely finished, and the batches table offers
"Retry undo" for that status. The column was String(16) and the new value is
exactly 16 characters, so migration 7d38 widens it to 32 rather than rely on an
exact fit.

_UNDO_REFERENCES listed only assets, so a business unit this batch created was
hard-deleted even when something else still pointed at it. Two of the three
missing referrers are real FKs (dashboarddefaults.businessunitid,
businessunits.parentid): MySQL raises and the whole undo rolls back, so the
batch cannot be undone at all. The third, notifications.businessunitid, has no
FK, so the delete succeeded and left a notification pointing at an id that no
longer exists - silently, on both engines. All four are listed and the guard is
tested through the route.

The reference guard also queried tables that need not exist. A lean site drops
the tables of plugins it does not install (ADR-014), and notifications is
plugin-owned, so on such a site the query raised 1146, rolled the undo back and
left the batch permanently un-undoable. Absent tables are skipped; a table that
does not exist cannot hold a reference.

_PACKET_GROUPS reports on location, but apply_plan only ever creates business
units - it sets locationid only when the location already resolved. A plan
needing a new location asked the admin to acknowledge it, got the tick, wrote
nothing, and reported nothing. build_packet now takes the groups the writer can
actually create; anything else needed is 'uncreatable' and blocks at plan time,
where the admin can act on it, rather than dying at commit on a message about
acknowledging something the screen had no control for. The writer refuses the
same case independently. Both the console packet and the import page name the
blocking groups: setting blocking without printing the reason would leave the
banner pointing at nothing.

The web undo route caught ValueError only, so refusing a batch belonging to
another importer surfaced as a 500. That refusal is new too: the journal is a
general primitive and undo_batch does not filter on kind, so any other
importer's batches were reversible through this endpoint, carrying WINC's
references map - which lists none of that importer's tables, making every
reference check pass and every delete unconditional.

Nine tests. The two guards were verified by planting violations: dropping
dashboarddefaults from the references map, and passing creatable=None, each fail
the test written for them.

Scope: product

scripts/site_imports/winc/writer.py

@@ -51,17 +51,40 @@ _PACKET_GROUPS = [
     ('location', 'location_status', 'raw_location', 'location_name'),
 ]
 
+# Groups apply_plan can actually create. _PACKET_GROUPS is what the plan
+# REPORTS on; this is what the writer can act on, and the two are not the same.
+_CREATABLE_GROUPS = ('businessunit',)
+
 # References into a created row's id, so undo can tell whether an inserted
 # business unit is still pointed at by something outside this batch before it
 # hard-deletes it. Shared with the undo CLI and the web undo path.
 _UNDO_REFERENCES = {
-    'businessunits': [('assets', 'businessunitid')],
+    # EVERY referrer, not just the obvious one. Undo hard-deletes a business
+    # unit this batch created, and it may only do that when nothing outside the
+    # batch still points at it. assets was the only entry, so a BU referenced
+    # from anywhere else was deleted anyway:
+    #
+    #   - dashboarddefaults.businessunitid and businessunits.parentid are real
+    #     FKs, so MySQL raises and the whole undo rolls back - consistent, but
+    #     the batch then cannot be undone at all;
+    #   - notifications.businessunitid has NO FK, so the delete succeeded and
+    #     left the notification pointing at an id that no longer exists.
+    #     Silently, on both engines.
+    #
+    # The guard itself is a SELECT COUNT(*), not FK enforcement, so it runs the
+    # same on SQLite - a wrong-but-real column name here would silently match
+    # nothing and delete the row anyway, which is why it is tested rather than
+    # trusted. What SQLite cannot show is the CONSEQUENCE of getting it wrong:
+    # MySQL raises on the FK cases, SQLite lets the dangling row through.
+    'businessunits': [
+        ('assets', 'businessunitid'),
+        ('dashboarddefaults', 'businessunitid'),
+        ('businessunits', 'parentid'),
+        ('notifications', 'businessunitid'),
+    ],
 }
 
 
-# References into a created row's id, so undo can tell whether an inserted
-# business unit is still pointed at by something outside this batch before it
-# hard-deletes it.
 # Max width of assets.name; the export description can exceed it (a few rows
 # carry a whole safety notice in the field), so a set name is clamped rather
 # than letting the whole transaction fail on one over-long row. The pre-image is
@@ -148,7 +171,8 @@ def apply_plan(connection, plan, approved_packet, acknowledge=None,
     # (2)-(3) rebuild the packet and prove nothing drifted since approval.
     current_packet = build_packet('winc-machine-adoption', plan,
                                   approved_packet.get('counts', {}),
-                                  _PACKET_GROUPS)
+                                  _PACKET_GROUPS,
+                                  creatable=_CREATABLE_GROUPS)
     if not packet_matches_plan(approved_packet, plan):
         raise WriterError(
             'plan has changed since approval (hash mismatch) - re-run the '
@@ -167,6 +191,24 @@ def apply_plan(connection, plan, approved_packet, acknowledge=None,
             'needs-create not acknowledged for: %s - confirm these will be '
             'created before importing' % ', '.join(sorted(missing_ack)))
 
+    # (5b) and the writer must actually be able to create what it just made the
+    # admin agree to. _PACKET_GROUPS lists location, so a plan needing a new
+    # location asked for acknowledgement - and then step 6a creates business
+    # units only, and 6b sets locationid only when the location already
+    # RESOLVED. The admin was told "this would create 3 locations", ticked it,
+    # and nothing was created, nothing was set, and nothing was reported.
+    #
+    # Refusing is the honest answer: better a plan that will not run than one
+    # that quietly does less than it said. Creating locations is a real feature
+    # and can be added here; until then this cannot pretend.
+    uncreatable = (set(current_packet.get('needscreate', {}))
+                   - set(_CREATABLE_GROUPS))
+    if uncreatable:
+        raise WriterError(
+            'this import cannot create: %s - the rows must exist before it '
+            'runs, or be aliased to rows that do. Nothing was written.'
+            % ', '.join(sorted(uncreatable)))
+
     batchid = approved_packet['batchid']
     journal.open_batch(connection, approved_packet, createdby=createdby)
     sequence = 1
@@ -244,7 +286,7 @@ def main(argv=None):
     _app, connection = build_app_connection()
     plan, summary, _lookups = analyze(connection, args.export)
     packet = build_packet('winc-machine-adoption', plan, summary,
-                          _PACKET_GROUPS)
+                          _PACKET_GROUPS, creatable=_CREATABLE_GROUPS)
 
     if not args.commit:
         print('DRY: rebuilt plan of %d rows, hash %s' % (