Gate a restore on every schema chain, not just the core one
Some checks failed
CI / naming (push) Successful in 4s
CI / frontend (push) Successful in 13s
CI / migrations-mysql (push) Failing after 15s
CI / backend (push) Failing after 4m56s

The compatibility gate read alembic_version and nothing else, but plugin schemas
move independently of core. Roll the code back to an older plugin revision and
the core head can be identical, so a dump carrying plugin tables the code has
never heard of passed the gate, loaded - destroying what was there - and only
then failed in 'flask plugin upgrade-all' with "Can't locate revision". A failed
restore that has already replaced what it was restoring over is the worst
outcome this feature can produce, and it was reachable from an ordinary rollback.

Backups now stamp the applied revision of every alembic_version_<plugin> table
alongside the core one, read from the dumped database the same way. restore.sh
checks each against that plugin's own versions directory and refuses before
touching anything, naming the plugin and the revision to deploy past. A plugin
in the dump that this build does not have at all is refused for the same reason.
The list endpoint's compatibility badge asks the same question, so the UI cannot
offer a set the restore will reject.

Sets taken before the stamp existed carry no dataplugins and keep their old
core-only verdict. Their plugin revisions are unknowable, and making every
existing set unrestorable to close this would be a worse trade than the gap.

Proven against a real stack, reproducing the exact case: core head identical,
endpoints data stamped at a revision the build does not have. The restore
refused, the live row was still there afterwards, and it stopped early enough
that the pre-restore safety copy was never needed. The same set with its true
revision restored normally.

Suite 2605 passed / 9 skipped, with the gate covered both ways plus the legacy
fallback.

Scope: product
This commit is contained in:
cproudlock
2026-09-06 18:02:39 -04:00
parent fa6b99bf91
commit e2705d732a
6 changed files with 246 additions and 6 deletions

View File

@@ -383,6 +383,63 @@ def _known_core_revisions():
return set()
def _known_plugin_revisions():
"""Every revision each plugin chain knows, as {plugin: {revisions}}.
Read by text scan rather than through Alembic. Each plugin chain has its own
env and version table, so loading them properly would mean building 17
ScriptDirectories on a request that only needs to answer "is this id in
there" - and manifest.py already reads the same trees the same way, so the
two agree by construction.
"""
root = os.path.dirname(current_app.root_path)
plugins_dir = os.path.join(root, 'plugins')
known = {}
if not os.path.isdir(plugins_dir):
return known
for name in sorted(os.listdir(plugins_dir)):
versions = os.path.join(plugins_dir, name, 'migrations', 'versions')
if not os.path.isdir(versions):
continue
revisions = set()
for filename in os.listdir(versions):
if not filename.endswith('.py'):
continue
try:
with open(os.path.join(versions, filename), 'r',
encoding='utf-8') as handle:
for line in handle:
stripped = line.strip()
if stripped.startswith('revision ='):
revisions.add(
stripped.split('=', 1)[1].strip().strip('"\''))
break
except OSError:
continue
if revisions:
known[name] = revisions
return known
def _plugin_data_is_known(manifest):
"""Whether this build knows every plugin revision the set's data sits at.
A set with no `dataplugins` predates the stamp; its plugin revisions are
unknowable, so it keeps the old core-only verdict rather than being marked
unrestorable retroactively.
"""
stamped = manifest.get('dataplugins') or {}
if not isinstance(stamped, dict) or not stamped:
return True
known = _known_plugin_revisions()
for name, revision in stamped.items():
if not revision:
continue
if revision not in known.get(name, ()):
return False
return True
def _is_compatible(manifest, current):
"""Decide whether a set restores onto THIS build.
@@ -395,6 +452,11 @@ def _is_compatible(manifest, current):
revision, the dump is from a NEWER schema than this build and would 500 on
restored rows, so we block until the code is upgraded.
The same question is asked of every PLUGIN chain, because plugin schemas
move independently: code rolled back to an older plugin revision can sit at
an identical core head while the dump carries plugin tables it cannot
migrate.
Legacy sets predate `datarev`; fall back to the old head-subset check on
`schemarev` so they stay listable and restorable as before.
"""
@@ -405,7 +467,10 @@ def _is_compatible(manifest, current):
known = _known_core_revisions()
if not known:
return False
return datarev in known
# Core AND every plugin chain. Core alone said yes to a set whose plugin
# data the code had never heard of, which loads and then dies during the
# plugin upgrade with the old data already gone.
return datarev in known and _plugin_data_is_known(manifest)
setrevs = set(manifest.get('schemarev', {}).get('core', []))
currevs = set(current.get('core', []))
if not setrevs or not currevs: