ADR-013 Phase 2: enforcement + signed shelf + adopt

Completes the marketplace security model. Verification stops being advisory:
a plugin only loads or migrates when its tree matches a trusted signature, and
plugins are pulled from a signed shelf with anti-rollback and revocation.

Enforcement (default OFF - existing deploys unchanged):
- verification.py PluginVerifier, shared by the loader (verify-at-load, before
  plugin.py is imported) and the migration manager (verify-at-migrate, before
  any DDL). Fail-closed: an unsigned/tampered/wrong-key plugin does not run.
- Gated by PLUGIN_REQUIRE_SIGNED. PLUGIN_DEV_TRUST_DIRS exempts named dirs but
  only under DEBUG/TESTING; production ignores it.
- flask plugin stamp-bundled writes provenance into in-tree plugins so
  verify-at-load applies to bundled plugins too (image build step).
- tier:core manifest guard: uninstall/disable refuse a core-tier plugin.

Shelf (shelf.py):
- Signed shelf-index.json (+ .sig): monotonic serial (a site refuses an older
  index - anti-rollback), revoked list carried across builds, per-entry
  version/tier/core_version for browse. Index is a browse layer only; adopt
  reads security-bearing fields from the verified artifact.
- flask plugin shelf-build / shelf-list / adopt / audit. adopt verifies index +
  artifact (signature + every file hash), unpacks to staging, re-verifies, then
  atomically moves into place and installs+enables the closure. Refuses a
  downgrade without --force-downgrade. Anti-rollback serial stored in
  instance/shelf-state.json.
- config PLUGIN_SHELF_DIR; the app only reads the folder, never speaks a
  network. .env.example + docs/PLUGIN-SIGNING.md document the flow.

22 tests: verifier policy (off / no-keys / signed / tampered / wrong-key /
dev-exempt), verify-at-load + verify-at-migrate integration, tier guard, index
sign/verify + tamper/wrong-key, serial state, revocation, version resolution,
verified atomic unpack + tamper refusal. Live-smoked keygen->pack->shelf-build
->list->adopt->audit + serial guard. 1050 pass, naming green.
This commit is contained in:
cproudlock
2026-07-18 20:44:54 -04:00
parent 86f5f1be68
commit 5b19f3b554
12 changed files with 1048 additions and 8 deletions

View File

@@ -66,6 +66,19 @@ ZABBIX_TOKEN=
# to verify signed plugin artifacts. Delivered with this config, NEVER from the # to verify signed plugin artifacts. Delivered with this config, NEVER from the
# plugin shelf. Empty on a site that does not adopt marketplace plugins. # plugin shelf. Empty on a site that does not adopt marketplace plugins.
# PLUGIN_TRUSTED_KEYS=/etc/shopdb/keys/curator.pub # PLUGIN_TRUSTED_KEYS=/etc/shopdb/keys/curator.pub
#
# Enforce signatures: a plugin only loads/migrates if its tree matches a
# trusted signature. Default off. Turn on only after stamping plugins
# (flask plugin stamp-bundled) and pinning keys above.
# PLUGIN_REQUIRE_SIGNED=false
#
# Dev-only: directories whose UNSIGNED plugins are trusted, honored ONLY under
# DEBUG/TESTING (external-repo/symlink dev). Production ignores this.
# PLUGIN_DEV_TRUST_DIRS=/home/dev/my-plugin-repo
#
# Read-only folder the app pulls plugin artifacts from (a SharePoint-synced or
# copied shelf). The app reads this folder; it never speaks any network.
# PLUGIN_SHELF_DIR=/srv/shopdb/plugin-shelf
# ---- Employee directory database (optional, read-only) ---- # ---- Employee directory database (optional, read-only) ----
# Separate HR/employee lookup DB consumed by the notifications plugin and the # Separate HR/employee lookup DB consumed by the notifications plugin and the

View File

@@ -67,9 +67,60 @@ Keys are read only from this deployed config, never from the shelf - a folder an
attacker could write must not also carry the keys that authenticate it. With no attacker could write must not also carry the keys that authenticate it. With no
keys set, `validate` on an artifact fails closed (unverifiable). keys set, `validate` on an artifact fails closed (unverifiable).
## What is NOT in this phase ## Enforce signatures (Phase 2)
Verification is available but not yet enforced automatically at plugin load or By default nothing is enforced - plugins load unsigned, as before. To require
migrate time, and there is no `adopt` command that pulls from a shelf yet. Those signatures on a site:
land in Phase 2 (verify-at-load, verify-at-migrate, the signed shelf index).
For now, verify artifacts manually with `validate` before installing. 1. Stamp the plugins the image ships with, so verify-at-load applies to them
too (run at image build with the site/build key):
```
flask plugin stamp-bundled --key ./keys/curator.key
```
This writes `PROVENANCE.json` + `PROVENANCE.sig` into each in-tree plugin.
2. Pin the public key(s) and turn enforcement on (site config):
```
PLUGIN_TRUSTED_KEYS=/etc/shopdb/keys/curator.pub
PLUGIN_REQUIRE_SIGNED=true
```
Now a plugin only loads (verify-at-load) or migrates (verify-at-migrate) when
its tree matches a trusted signature. An unsigned, tampered, or wrong-key
plugin is refused - fail-closed. `PLUGIN_DEV_TRUST_DIRS` exempts named
directories, but ONLY under DEBUG/TESTING (the external-repo dev workflow);
production ignores it.
## The shelf and adopt (Phase 2)
A shelf is a read-only folder of artifacts plus a signed index. The app reads
`PLUGIN_SHELF_DIR`; it never talks to SharePoint - a sync (or robocopy/USB)
populates that folder, and the signature makes the transport untrusted and
interchangeable.
Publish (curator, after packing artifacts into the shelf folder):
```
flask plugin shelf-build --dir /srv/shelf --key ./keys/curator.key --serial 3
```
The index carries a monotonic `serial` (a site refuses an index older than the
last it saw) and a `revoked` list (carried forward across builds). Bump
`--serial` on every publish.
On a site:
```
flask plugin shelf-list # browse (verifies index + serial)
flask plugin adopt printers # or printers==1.2.0
flask plugin audit # warn if an installed version is revoked
```
`adopt` verifies the shelf index and the artifact (signature + every file
hash), unpacks into a staging area, re-verifies, then atomically moves it into
place and installs + enables it with its dependency closure. It refuses a
downgrade unless `--force-downgrade`. Run `flask plugin upgrade-all` and restart
afterward so migrations apply and routes register.

View File

@@ -70,6 +70,25 @@ class Config:
if path.strip() if path.strip()
] ]
# When true, a plugin only loads/migrates if its tree matches a trusted
# signature (verify-at-load, verify-at-migrate). Default false = existing
# behavior. Turn on only after stamping plugins + pinning keys.
PLUGIN_REQUIRE_SIGNED = os.environ.get(
'PLUGIN_REQUIRE_SIGNED', 'false').lower() == 'true'
# Directories whose unsigned plugins are trusted - honored ONLY under
# DEBUG/TESTING (the external-repo/symlink dev workflow). Production ignores.
PLUGIN_DEV_TRUST_DIRS = [
path.strip()
for path in os.environ.get('PLUGIN_DEV_TRUST_DIRS', '').split(os.pathsep)
if path.strip()
]
# Read-only folder the app pulls plugin artifacts from (a SharePoint-synced
# or copied shelf). Empty = no shelf configured. The app never speaks any
# network protocol; it reads this folder.
PLUGIN_SHELF_DIR = os.environ.get('PLUGIN_SHELF_DIR', '')
ZABBIX_ENABLED = os.environ.get('ZABBIX_ENABLED', 'false').lower() == 'true' ZABBIX_ENABLED = os.environ.get('ZABBIX_ENABLED', 'false').lower() == 'true'
ZABBIX_URL = os.environ.get('ZABBIX_URL', '') ZABBIX_URL = os.environ.get('ZABBIX_URL', '')
ZABBIX_TOKEN = os.environ.get('ZABBIX_TOKEN', '') ZABBIX_TOKEN = os.environ.get('ZABBIX_TOKEN', '')

View File

@@ -39,6 +39,7 @@ class PluginManager:
self.registry: Optional[PluginRegistry] = None self.registry: Optional[PluginRegistry] = None
self.loader: Optional[PluginLoader] = None self.loader: Optional[PluginLoader] = None
self.migration_manager: Optional[PluginMigrationManager] = None self.migration_manager: Optional[PluginMigrationManager] = None
self.verifier = None
self._app: Optional[Flask] = None self._app: Optional[Flask] = None
self._db = None self._db = None
# API prefixes already claimed by a registered plugin blueprint, to # API prefixes already claimed by a registered plugin blueprint, to
@@ -66,6 +67,19 @@ class PluginManager:
app.config.get('SQLALCHEMY_DATABASE_URI') app.config.get('SQLALCHEMY_DATABASE_URI')
) )
# One trust policy, shared by verify-at-load and verify-at-migrate.
from .verification import PluginVerifier
verifier = PluginVerifier(
plugins_dir,
require_signed=app.config.get('PLUGIN_REQUIRE_SIGNED', False),
trusted_key_paths=app.config.get('PLUGIN_TRUSTED_KEYS', []),
dev_trust_dirs=app.config.get('PLUGIN_DEV_TRUST_DIRS', []),
is_dev=bool(app.config.get('DEBUG') or app.config.get('TESTING')),
)
self.verifier = verifier
self.loader.verifier = verifier
self.migration_manager.verifier = verifier
# Load enabled plugins # Load enabled plugins
self._load_enabled_plugins() self._load_enabled_plugins()
@@ -257,6 +271,18 @@ class PluginManager:
"Seeded %d permission(s) for plugin %s", "Seeded %d permission(s) for plugin %s",
created, plugin.meta.name) created, plugin.meta.name)
def _is_core_tier(self, name: str) -> bool:
"""True when the plugin's manifest marks it tier=core (mandatory).
A core-tier plugin refuses uninstall/disable so a site cannot remove a
plugin its deployment depends on. No plugin ships tier=core today; the
guard makes that a manifest edit, not a code change (ADR-013).
"""
try:
return self.loader.load_manifest(name).get('tier') == 'core'
except PluginError:
return False
def _installed_dependents(self, name: str, def _installed_dependents(self, name: str,
enabled_only: bool = False) -> List[str]: enabled_only: bool = False) -> List[str]:
"""Installed plugins that declare `name` as a hard dependency. """Installed plugins that declare `name` as a hard dependency.
@@ -293,6 +319,10 @@ class PluginManager:
logger.warning(f"Plugin {name} is not installed") logger.warning(f"Plugin {name} is not installed")
return False return False
if self._is_core_tier(name):
logger.error(f"Cannot uninstall {name}: it is a core-tier plugin")
return False
# Refuse if any INSTALLED plugin (enabled or not) depends on this one. # Refuse if any INSTALLED plugin (enabled or not) depends on this one.
dependents = self._installed_dependents(name) dependents = self._installed_dependents(name)
if dependents: if dependents:
@@ -372,6 +402,10 @@ class PluginManager:
logger.info(f"Plugin {name} is already disabled") logger.info(f"Plugin {name} is already disabled")
return True return True
if self._is_core_tier(name):
logger.error(f"Cannot disable {name}: it is a core-tier plugin")
return False
# Refuse if any ENABLED plugin depends on this one (a disabled dependent # Refuse if any ENABLED plugin depends on this one (a disabled dependent
# is not running, so it does not block). Manifests read from disk. # is not running, so it does not block). Manifests read from disk.
dependents = self._installed_dependents(name, enabled_only=True) dependents = self._installed_dependents(name, enabled_only=True)

View File

@@ -553,6 +553,283 @@ def apply_profile(profile: str):
"restart so new blueprints/routes register.", fg='yellow')) "restart so new blueprints/routes register.", fg='yellow'))
def _shelf_keys(pubkeys):
"""Trusted keys for shelf/artifact verification: explicit --pubkey first,
else the site's PLUGIN_TRUSTED_KEYS."""
from .signing import load_trusted_keys
paths = list(pubkeys) or current_app.config.get('PLUGIN_TRUSTED_KEYS', [])
return load_trusted_keys(paths), paths
@plugin_cli.command('stamp-bundled')
@click.option('--key', 'key_path', required=True, type=click.Path(exists=True),
help='Private signing key PEM')
@click.option('--publisher', default='', help='Publisher id in the provenance')
@click.argument('name', required=False)
@with_appcontext
def stamp_bundled(key_path, publisher, name):
"""Write PROVENANCE.json/.sig into in-tree plugin dirs (image build step).
Stamps one plugin (NAME) or every discovered plugin, so verify-at-load
applies to bundled plugins identically to adopted ones. Run at image build
with the site/build key, then set PLUGIN_REQUIRE_SIGNED.
"""
from .signing import (load_private_key, build_provenance,
serialize_provenance, sign,
PROVENANCE_NAME, PROVENANCE_SIG)
pm = current_app.extensions.get('plugin_manager')
if not pm:
click.echo(click.style("Plugin manager not initialized", fg='red'))
raise SystemExit(1)
private_key = load_private_key(Path(key_path).read_bytes())
names = [name] if name else pm.loader.discover_plugins()
plugins_dir = Path(pm.loader.plugins_dir)
for plugin_name in names:
plugin_dir = plugins_dir / plugin_name
manifest = pm.loader.load_manifest(plugin_name)
provenance = build_provenance(
plugin_dir, manifest['name'], manifest['version'], publisher)
provenance_bytes = serialize_provenance(provenance)
(plugin_dir / PROVENANCE_NAME).write_bytes(provenance_bytes)
(plugin_dir / PROVENANCE_SIG).write_bytes(
sign(private_key, provenance_bytes))
click.echo(click.style(f" stamped {plugin_name}", fg='green'))
@plugin_cli.command('shelf-build')
@click.option('--dir', 'shelf_dir', required=True,
type=click.Path(exists=True, file_okay=False),
help='Shelf directory holding the artifacts')
@click.option('--key', 'key_path', required=True,
type=click.Path(exists=True), help='Private signing key PEM')
@click.option('--serial', type=int, required=True,
help='Index serial (must exceed the current published serial)')
@click.option('--publisher', default='')
def shelf_build(shelf_dir, key_path, serial, publisher):
"""Build + sign the shelf index from the artifacts in DIR.
Carries forward the existing revoked list and refuses a serial that does not
advance past the published one (a lower serial would let a site's
anti-rollback reject the new index).
"""
import json as _json
from .signing import load_private_key
from .shelf import build_index, INDEX_NAME
shelf = Path(shelf_dir)
revoked = []
existing = shelf / INDEX_NAME
if existing.exists():
try:
prev = _json.loads(existing.read_text())
revoked = prev.get('revoked', [])
if serial <= int(prev.get('serial', -1)):
click.echo(click.style(
f"serial {serial} must exceed current "
f"{prev.get('serial')}", fg='red'))
raise SystemExit(1)
except (ValueError, TypeError):
pass
private_key = load_private_key(Path(key_path).read_bytes())
index = build_index(shelf, private_key, serial,
publisher=publisher, revoked=revoked)
total = sum(len(v) for v in index['plugins'].values())
click.echo(click.style(
f"Shelf index signed: serial {serial}, {total} artifact(s), "
f"{len(revoked)} revoked.", fg='green'))
@plugin_cli.command('shelf-list')
@click.option('--pubkey', 'pubkeys', multiple=True, type=click.Path(exists=True))
@with_appcontext
def shelf_list(pubkeys):
"""Browse the configured shelf (verifies index signature + serial)."""
from shopdb import __contract_version__
from packaging.specifiers import SpecifierSet
from packaging.version import Version
from .shelf import load_index, read_last_serial, write_last_serial, is_revoked
shelf_dir = current_app.config.get('PLUGIN_SHELF_DIR')
if not shelf_dir:
click.echo(click.style("PLUGIN_SHELF_DIR not configured", fg='red'))
raise SystemExit(1)
keys, _ = _shelf_keys(pubkeys)
index, errors = load_index(shelf_dir, keys)
for err in errors:
click.echo(click.style(f" {err}", fg='red'))
if index is None:
raise SystemExit(1)
last = read_last_serial(current_app.instance_path)
if index['serial'] < last:
click.echo(click.style(
f"REFUSING shelf: index serial {index['serial']} < last seen "
f"{last} (rollback?)", fg='red'))
raise SystemExit(1)
if errors:
raise SystemExit(1)
write_last_serial(current_app.instance_path, index['serial'])
click.echo("")
click.echo(click.style(
f"Shelf (serial {index['serial']}):", fg='cyan', bold=True))
for name in sorted(index.get('plugins', {})):
for entry in index['plugins'][name]:
compat = 'ok'
spec = entry.get('core_version', '')
if spec and Version(__contract_version__) not in SpecifierSet(spec):
compat = f'needs core {spec}'
flags = []
if is_revoked(index, name, entry['version']):
flags.append('REVOKED')
if compat != 'ok':
flags.append(compat)
tag = (' [' + ', '.join(flags) + ']') if flags else ''
click.echo(f" {name:20} {entry['version']:10} "
f"({entry.get('tier', 'optional')}){tag}")
@plugin_cli.command('adopt')
@click.argument('spec')
@click.option('--pubkey', 'pubkeys', multiple=True, type=click.Path(exists=True))
@click.option('--force-downgrade', is_flag=True,
help='Allow adopting a version lower than the installed one')
@with_appcontext
def adopt_plugin(spec, pubkeys, force_downgrade):
"""Pull a plugin from the shelf, verify it, and install + enable it.
SPEC is a plugin name or name==version. Verifies the shelf index and the
artifact signature + hashes, unpacks atomically only after verification,
then installs + enables the plugin and its dependency closure.
Usage: flask plugin adopt printers (or printers==1.2.0)
"""
from packaging.version import Version
from ..exceptions import PluginError
from .shelf import (load_index, read_last_serial, write_last_serial,
resolve_version, unpack_verified)
pm = current_app.extensions.get('plugin_manager')
if not pm:
click.echo(click.style("Plugin manager not initialized", fg='red'))
raise SystemExit(1)
name, _, version = spec.partition('==')
name = name.strip()
version = version.strip() or None
shelf_dir = current_app.config.get('PLUGIN_SHELF_DIR')
if not shelf_dir:
click.echo(click.style("PLUGIN_SHELF_DIR not configured", fg='red'))
raise SystemExit(1)
keys, _ = _shelf_keys(pubkeys)
index, errors = load_index(shelf_dir, keys)
for err in errors:
click.echo(click.style(f" {err}", fg='red'))
if index is None or errors:
raise SystemExit(1)
last = read_last_serial(current_app.instance_path)
if index['serial'] < last:
click.echo(click.style(
f"REFUSING shelf: serial {index['serial']} < last seen {last} "
f"(rollback?)", fg='red'))
raise SystemExit(1)
try:
resolved_version, artifact_name = resolve_version(index, name, version)
except (KeyError, ValueError) as e:
click.echo(click.style(f"{e}", fg='red'))
raise SystemExit(1)
state = pm.registry.get(name)
if state and not force_downgrade:
try:
if Version(resolved_version) < Version(state.version):
click.echo(click.style(
f"Refusing downgrade {state.version} -> {resolved_version} "
f"(use --force-downgrade)", fg='red'))
raise SystemExit(1)
except Exception:
pass
artifact_path = Path(shelf_dir) / artifact_name
plugins_dir = Path(pm.loader.plugins_dir)
click.echo(f"Verifying + unpacking {name} {resolved_version} ...")
try:
unpack_verified(artifact_path, keys, plugins_dir, name)
except ValueError as e:
click.echo(click.style(f" verification failed: {e}", fg='red'))
raise SystemExit(1)
# Clear any cached manifest so the freshly unpacked one is read.
pm.loader._manifests.pop(name, None)
try:
result = pm.apply_profile([name])
except PluginError as e:
click.echo(click.style(f" {e}", fg='red'))
click.echo(click.style(
" (adopt the missing dependency plugins first)", fg='yellow'))
raise SystemExit(1)
write_last_serial(current_app.instance_path, index['serial'])
click.echo("")
click.echo(click.style(
f"Adopted {name} {resolved_version}.", fg='green'))
if result['installed']:
click.echo(f" installed: {', '.join(result['installed'])}")
if result['enabled']:
click.echo(f" enabled: {', '.join(result['enabled'])}")
click.echo(click.style(
"Run 'flask plugin upgrade-all' then restart so routes register.",
fg='yellow'))
@plugin_cli.command('audit')
@click.option('--pubkey', 'pubkeys', multiple=True, type=click.Path(exists=True))
@with_appcontext
def audit_plugins(pubkeys):
"""Warn about installed plugins whose version is revoked on the shelf."""
from .shelf import load_index, is_revoked
pm = current_app.extensions.get('plugin_manager')
if not pm:
click.echo(click.style("Plugin manager not initialized", fg='red'))
raise SystemExit(1)
shelf_dir = current_app.config.get('PLUGIN_SHELF_DIR')
if not shelf_dir:
click.echo(click.style("PLUGIN_SHELF_DIR not configured", fg='red'))
raise SystemExit(1)
keys, _ = _shelf_keys(pubkeys)
index, errors = load_index(shelf_dir, keys)
for err in errors:
click.echo(click.style(f" {err}", fg='red'))
if index is None:
raise SystemExit(1)
flagged = []
for name, state in pm.registry.get_all().items():
if is_revoked(index, name, state.version):
flagged.append(f"{name} {state.version}")
click.echo("")
if flagged:
click.echo(click.style(
"REVOKED plugins installed:", fg='red', bold=True))
for item in flagged:
click.echo(click.style(f" {item}", fg='red'))
raise SystemExit(1)
click.echo(click.style("No installed plugin is revoked.", fg='green'))
@plugin_cli.command('migrate') @plugin_cli.command('migrate')
@click.argument('name') @click.argument('name')
@click.option('--revision', default='head', help='Target revision') @click.option('--revision', default='head', help='Target revision')

View File

@@ -45,6 +45,8 @@ class PluginLoader:
self._loaded_plugins: Dict[str, BasePlugin] = {} self._loaded_plugins: Dict[str, BasePlugin] = {}
self._plugin_classes: Dict[str, Type[BasePlugin]] = {} self._plugin_classes: Dict[str, Type[BasePlugin]] = {}
self._manifests: Dict[str, dict] = {} self._manifests: Dict[str, dict] = {}
# Set by PluginManager.init_app; a PluginVerifier or None (no policy).
self.verifier = None
def _is_strict_mode(self, app: Optional[Flask]) -> bool: def _is_strict_mode(self, app: Optional[Flask]) -> bool:
"""Return True if loader should re-raise instead of isolating failures.""" """Return True if loader should re-raise instead of isolating failures."""
@@ -198,6 +200,17 @@ class PluginLoader:
manifest = self.load_manifest(name) manifest = self.load_manifest(name)
self.check_contract_version(name, __contract_version__) self.check_contract_version(name, __contract_version__)
# verify-at-load: refuse to import plugin.py unless the tree matches
# a trusted signature (when the site enforces it). Runs BEFORE any
# plugin code is imported, so a tampered/unsigned plugin never runs.
if self.verifier is not None:
ok, reason = self.verifier.check(name)
if not ok:
raise PluginError(
f'Plugin {name} failed signature verification: {reason}',
plugin_name=name,
)
for dep in manifest.get('dependencies', []): for dep in manifest.get('dependencies', []):
if not self.registry.is_enabled(dep): if not self.registry.is_enabled(dep):
raise PluginDependencyError( raise PluginDependencyError(

View File

@@ -18,6 +18,8 @@ class PluginMigrationManager:
def __init__(self, plugins_dir: Path, database_url: str): def __init__(self, plugins_dir: Path, database_url: str):
self.plugins_dir = plugins_dir self.plugins_dir = plugins_dir
self.database_url = database_url self.database_url = database_url
# Set by PluginManager.init_app; a PluginVerifier or None (no policy).
self.verifier = None
def get_migrations_dir(self, plugin_name: str) -> Optional[Path]: def get_migrations_dir(self, plugin_name: str) -> Optional[Path]:
"""Get migrations directory for a plugin.""" """Get migrations directory for a plugin."""
@@ -36,6 +38,16 @@ class PluginMigrationManager:
Uses flask db upgrade with the plugin's migrations directory. Uses flask db upgrade with the plugin's migrations directory.
""" """
# verify-at-migrate: never run a plugin's DDL (full DB rights) from an
# unverified tree when the site enforces signing.
if self.verifier is not None:
ok, reason = self.verifier.check(plugin_name)
if not ok:
logger.error(
"Refusing migrations for %s: signature verification failed "
"(%s)", plugin_name, reason)
return False
migrations_dir = self.get_migrations_dir(plugin_name) migrations_dir = self.get_migrations_dir(plugin_name)
if not migrations_dir: if not migrations_dir:

185
shopdb/plugins/shelf.py Normal file
View File

@@ -0,0 +1,185 @@
"""Signed plugin shelf: index build/verify, anti-rollback state, adopt mechanics
(ADR-013 Phase 2).
The shelf is a read-only folder of .shopdbplugin artifacts plus a signed
shelf-index.json (+ .sig). The index carries a monotonic serial (a site refuses
an index older than the last it saw) and a revoked list. Browse reads the index;
ADOPT always re-verifies the artifact's own signed manifest, never trusting the
index for anything security-bearing.
"""
import json
import shutil
import zipfile
from pathlib import Path
from packaging.version import Version, InvalidVersion
from . import signing, packaging
INDEX_NAME = 'shelf-index.json'
INDEX_SIG = 'shelf-index.sig'
STATE_NAME = 'shelf-state.json'
STAGING_DIR = '.staging'
def _read_artifact_manifest(artifact_path) -> dict:
with zipfile.ZipFile(artifact_path) as archive:
return json.loads(archive.read('manifest.json'))
def build_index(shelf_dir, private_key, serial: int,
publisher: str = '', revoked=None) -> dict:
"""Scan the shelf for artifacts, build and SIGN the index. Caller supplies a
serial that must exceed any previously published one."""
shelf_dir = Path(shelf_dir)
plugins = {}
for artifact in sorted(shelf_dir.glob(f'*{packaging.ARTIFACT_SUFFIX}')):
manifest = _read_artifact_manifest(artifact)
plugins.setdefault(manifest['name'], []).append({
'version': manifest['version'],
'tier': manifest.get('tier', 'optional'),
'core_version': manifest.get('core_version', ''),
'artifact': artifact.name,
})
index = {
'serial': int(serial),
'publisher': publisher or '',
'plugins': plugins,
'revoked': list(revoked or []),
}
index_bytes = signing.serialize_provenance(index) # canonical, sorted bytes
(shelf_dir / INDEX_NAME).write_bytes(index_bytes)
(shelf_dir / INDEX_SIG).write_bytes(signing.sign(private_key, index_bytes))
return index
def load_index(shelf_dir, public_keys):
"""(index_or_None, errors). Verifies the index signature. Serial-vs-state is
the caller's check (needs the instance path)."""
shelf_dir = Path(shelf_dir)
index_path = shelf_dir / INDEX_NAME
sig_path = shelf_dir / INDEX_SIG
if not index_path.exists() or not sig_path.exists():
return None, ['shelf has no signed index (shelf-index.json/.sig)']
index_bytes = index_path.read_bytes()
signature = sig_path.read_bytes()
errors = []
if not public_keys:
errors.append('no trusted keys - shelf index unverifiable')
elif not signing.verify(public_keys, index_bytes, signature):
errors.append('shelf index signature does not match any trusted key')
try:
index = json.loads(index_bytes)
except json.JSONDecodeError as exc:
return None, errors + [f'shelf index is not valid JSON: {exc}']
return index, errors
def read_last_serial(instance_path) -> int:
state = Path(instance_path) / STATE_NAME
if state.exists():
try:
return int(json.loads(state.read_text()).get('last_serial', -1))
except (json.JSONDecodeError, ValueError, TypeError):
return -1
return -1
def write_last_serial(instance_path, serial: int) -> None:
state = Path(instance_path) / STATE_NAME
state.parent.mkdir(parents=True, exist_ok=True)
state.write_text(json.dumps({'last_serial': int(serial)}, indent=2))
def is_revoked(index: dict, name: str, version: str) -> bool:
for entry in index.get('revoked', []):
if entry.get('name') == name and entry.get('version') == version:
return True
return False
def _safe_version(value):
try:
return Version(value)
except InvalidVersion:
return None
def resolve_version(index: dict, name: str, version: str = None):
"""Pick the artifact filename for name[==version] from the index.
With no version, picks the highest non-revoked version. Returns
(version, artifact_filename) or raises KeyError/ValueError.
"""
candidates = index.get('plugins', {}).get(name)
if not candidates:
raise KeyError(f'plugin {name} not on the shelf')
if version is not None:
for entry in candidates:
if entry['version'] == version:
if is_revoked(index, name, version):
raise ValueError(f'{name} {version} is revoked')
return version, entry['artifact']
raise KeyError(f'{name} {version} not on the shelf')
live = [e for e in candidates if not is_revoked(index, name, e['version'])]
if not live:
raise ValueError(f'every published version of {name} is revoked')
best = max(live, key=lambda e: (_safe_version(e['version']) or Version('0'),))
return best['version'], best['artifact']
def unpack_verified(artifact_path, public_keys, plugins_dir, name: str):
"""Verify an artifact, then atomically place it at plugins_dir/name.
Verifies signature + every file hash in a staging area BEFORE it can be
imported; only a fully verified tree is moved into place. Returns the final
plugin directory. Raises ValueError with all problems on any failure.
"""
plugins_dir = Path(plugins_dir)
manifest, errors = packaging.verify_artifact(artifact_path, public_keys)
if errors:
raise ValueError('; '.join(errors))
if manifest.get('name') != name:
raise ValueError(
f"artifact manifest name '{manifest.get('name')}' != '{name}'")
staging = plugins_dir / STAGING_DIR
staging.mkdir(parents=True, exist_ok=True)
staged = staging / name
if staged.exists():
shutil.rmtree(staged)
with zipfile.ZipFile(artifact_path) as archive:
archive.extractall(staged)
# Re-verify the unpacked tree before it goes live (defense against an
# extract that did not land what we verified).
_, staged_errors = packaging.verify_dir(staged, public_keys)
if staged_errors:
shutil.rmtree(staged, ignore_errors=True)
raise ValueError('staged tree failed re-verification: '
+ '; '.join(staged_errors))
target = plugins_dir / name
backup = staging / f'{name}.backup'
if target.exists():
if backup.exists():
shutil.rmtree(backup)
target.rename(backup)
try:
staged.rename(target)
except OSError:
# cross-device or rename race: fall back to copy, keep backup on failure
shutil.copytree(staged, target)
shutil.rmtree(staged, ignore_errors=True)
if backup.exists():
shutil.rmtree(backup, ignore_errors=True)
return target

View File

@@ -0,0 +1,64 @@
"""Plugin signature enforcement policy (ADR-013 Phase 2).
One PluginVerifier is built per app from config and shared by the loader
(verify-at-load) and the migration manager (verify-at-migrate). A plugin only
loads or migrates when it verifies against a trusted key - so signing stops
being advisory and becomes enforced everywhere plugin code executes.
Default is OFF (PLUGIN_REQUIRE_SIGNED unset), so existing deploys are
unchanged. A site opts into hardening by stamping its plugins
(`flask plugin stamp-bundled`), pinning keys (PLUGIN_TRUSTED_KEYS), and setting
PLUGIN_REQUIRE_SIGNED. Fail-closed: an unverifiable plugin does NOT run.
"""
import logging
from pathlib import Path
from . import signing, packaging
logger = logging.getLogger(__name__)
class PluginVerifier:
"""Decides whether a plugin may load/migrate under the site's trust policy."""
def __init__(self, plugins_dir, require_signed=False, trusted_key_paths=None,
dev_trust_dirs=None, is_dev=False):
self.plugins_dir = Path(plugins_dir)
self.require_signed = bool(require_signed)
self.is_dev = bool(is_dev)
# dev trust dirs only mean anything in dev/test (ADR-013); prod ignores.
self.dev_trust_dirs = [Path(p).resolve() for p in (dev_trust_dirs or [])]
self._keys = signing.load_trusted_keys(trusted_key_paths or [])
def _dev_exempt(self, plugin_dir: Path) -> bool:
# unsigned dev/external-repo plugins pass, but ONLY under DEBUG/TESTING
if not self.is_dev or not self.dev_trust_dirs:
return False
resolved = Path(plugin_dir).resolve()
for trusted in self.dev_trust_dirs:
try:
resolved.relative_to(trusted)
return True
except ValueError:
continue
return False
def check(self, plugin_name: str):
"""(ok, reason). ok True = plugin may load/migrate."""
if not self.require_signed:
return True, 'enforcement off'
plugin_dir = self.plugins_dir / plugin_name
if self._dev_exempt(plugin_dir):
return True, 'dev-trusted'
if not self._keys:
# require-signed with nothing to verify against = refuse everything.
return False, ('PLUGIN_REQUIRE_SIGNED is set but no trusted keys '
'are configured')
_, errors = packaging.verify_dir(plugin_dir, self._keys)
if errors:
return False, '; '.join(errors)
return True, 'verified'

View File

@@ -58,18 +58,21 @@ def test_registry_migrates_equipment_to_machines(tmp_path):
# --- Manager enable/disable dependency guards + permission seeding ----------- # --- Manager enable/disable dependency guards + permission seeding -----------
def _write_plugin(plugins_dir, name, deps=None, permissions=None): def _write_plugin(plugins_dir, name, deps=None, permissions=None, tier=None):
"""Write a minimal valid synthetic plugin (manifest.json + plugin.py).""" """Write a minimal valid synthetic plugin (manifest.json + plugin.py)."""
pdir = plugins_dir / name pdir = plugins_dir / name
pdir.mkdir() pdir.mkdir()
(pdir / 'manifest.json').write_text(json.dumps({ manifest = {
'name': name, 'name': name,
'version': '1.0.0', 'version': '1.0.0',
'description': f'synthetic {name}', 'description': f'synthetic {name}',
'dependencies': deps or [], 'dependencies': deps or [],
'core_version': '>=0.1.0,<1.0.0', 'core_version': '>=0.1.0,<1.0.0',
'api_prefix': f'/api/{name}', 'api_prefix': f'/api/{name}',
})) }
if tier:
manifest['tier'] = tier
(pdir / 'manifest.json').write_text(json.dumps(manifest))
(pdir / '__init__.py').write_text('') (pdir / '__init__.py').write_text('')
(pdir / 'plugin.py').write_text( (pdir / 'plugin.py').write_text(
'from shopdb.plugins.base import BasePlugin, PluginMeta\n\n' 'from shopdb.plugins.base import BasePlugin, PluginMeta\n\n'
@@ -286,6 +289,28 @@ def test_validate_bundled_plugin_passes(app):
assert 'valid' in result.output assert 'valid' in result.output
def test_core_tier_plugin_cannot_be_disabled_or_uninstalled(app, db, tmp_path):
"""A tier=core plugin refuses disable and uninstall (ADR-013)."""
plugins_dir = tmp_path / 'plugins'
plugins_dir.mkdir()
_write_plugin(plugins_dir, 'kernel', tier='core')
registry = PluginRegistry(tmp_path / 'plugins.json')
registry.register('kernel', '1.0.0', enabled=True)
pm = PluginManager()
pm._app = app
pm._db = db
pm.registry = registry
pm.loader = PluginLoader(plugins_dir, registry)
pm.migration_manager = None
assert pm.disable_plugin('kernel') is False
assert pm.registry.is_enabled('kernel')
assert pm.uninstall_plugin('kernel') is False
assert pm.registry.is_installed('kernel')
# --- Seed idempotency ------------------------------------------------------- # --- Seed idempotency -------------------------------------------------------
def test_seed_settings_is_idempotent(app, db): def test_seed_settings_is_idempotent(app, db):

180
tests/test_plugin_shelf.py Normal file
View File

@@ -0,0 +1,180 @@
"""Signed shelf index + adopt-mechanics tests (ADR-013 Phase 2).
Covers: a signed index round-trips and a tampered/wrong-key index is rejected;
serial anti-rollback state; revocation; version resolution; and unpack_verified
placing only a fully verified tree (refusing a tampered artifact).
"""
import json
import zipfile
import pytest
from shopdb.plugins import signing, packaging, shelf
def _write_plugin(plugin_dir, name='demo', version='1.0.0', tier=None):
plugin_dir.mkdir(parents=True, exist_ok=True)
manifest = {
'name': name, 'version': version, 'description': 'demo',
'dependencies': [], 'core_version': '>=0.1.0,<1.0.0',
'api_prefix': f'/api/{name}',
}
if tier:
manifest['tier'] = tier
(plugin_dir / 'manifest.json').write_text(json.dumps(manifest))
(plugin_dir / 'plugin.py').write_text('# demo\n')
return plugin_dir
@pytest.fixture
def keypair(tmp_path):
private_pem, public_pem = signing.generate_keypair()
return (signing.load_private_key(private_pem),
signing.load_public_key(public_pem))
def _build_shelf(tmp_path, private_key, specs, serial=1, revoked=None):
"""specs: list of (name, version). Packs each into a shelf dir + index."""
shelf_dir = tmp_path / 'shelf'
shelf_dir.mkdir(parents=True, exist_ok=True)
for name, version in specs:
src = tmp_path / 'src' / f'{name}-{version}'
_write_plugin(src, name=name, version=version)
packaging.pack(src, private_key, out_dir=shelf_dir)
shelf.build_index(shelf_dir, private_key, serial, revoked=revoked)
return shelf_dir
# --- index sign/verify ------------------------------------------------------
def test_index_round_trip(tmp_path, keypair):
priv, pub = keypair
shelf_dir = _build_shelf(tmp_path, priv, [('demo', '1.0.0')])
index, errors = shelf.load_index(shelf_dir, [pub])
assert errors == []
assert index['serial'] == 1
assert 'demo' in index['plugins']
def test_tampered_index_rejected(tmp_path, keypair):
priv, pub = keypair
shelf_dir = _build_shelf(tmp_path, priv, [('demo', '1.0.0')])
# change the index bytes but keep the old signature
path = shelf_dir / shelf.INDEX_NAME
data = json.loads(path.read_text())
data['serial'] = 999
path.write_bytes(signing.serialize_provenance(data))
_, errors = shelf.load_index(shelf_dir, [pub])
assert any('signature does not match' in e for e in errors)
def test_index_wrong_key_rejected(tmp_path, keypair):
priv, _ = keypair
shelf_dir = _build_shelf(tmp_path, priv, [('demo', '1.0.0')])
wrong_pub = signing.load_public_key(signing.generate_keypair()[1])
_, errors = shelf.load_index(shelf_dir, [wrong_pub])
assert any('signature does not match' in e for e in errors)
def test_index_no_keys_unverifiable(tmp_path, keypair):
priv, _ = keypair
shelf_dir = _build_shelf(tmp_path, priv, [('demo', '1.0.0')])
_, errors = shelf.load_index(shelf_dir, [])
assert any('unverifiable' in e for e in errors)
# --- anti-rollback state ----------------------------------------------------
def test_serial_state_round_trip(tmp_path):
assert shelf.read_last_serial(tmp_path) == -1
shelf.write_last_serial(tmp_path, 5)
assert shelf.read_last_serial(tmp_path) == 5
# --- revocation + version resolution ----------------------------------------
def test_resolve_highest_non_revoked(tmp_path, keypair):
priv, pub = keypair
shelf_dir = _build_shelf(
tmp_path, priv, [('demo', '1.0.0'), ('demo', '1.2.0')])
index, _ = shelf.load_index(shelf_dir, [pub])
version, artifact = shelf.resolve_version(index, 'demo')
assert version == '1.2.0'
assert artifact == 'demo-1.2.0.shopdbplugin'
def test_resolve_skips_revoked(tmp_path, keypair):
priv, pub = keypair
shelf_dir = _build_shelf(
tmp_path, priv, [('demo', '1.0.0'), ('demo', '1.2.0')],
revoked=[{'name': 'demo', 'version': '1.2.0'}])
index, _ = shelf.load_index(shelf_dir, [pub])
version, _ = shelf.resolve_version(index, 'demo')
assert version == '1.0.0'
def test_resolve_explicit_revoked_raises(tmp_path, keypair):
priv, pub = keypair
shelf_dir = _build_shelf(
tmp_path, priv, [('demo', '1.0.0')],
revoked=[{'name': 'demo', 'version': '1.0.0'}])
index, _ = shelf.load_index(shelf_dir, [pub])
with pytest.raises(ValueError):
shelf.resolve_version(index, 'demo', '1.0.0')
def test_resolve_unknown_raises(tmp_path, keypair):
priv, pub = keypair
shelf_dir = _build_shelf(tmp_path, priv, [('demo', '1.0.0')])
index, _ = shelf.load_index(shelf_dir, [pub])
with pytest.raises(KeyError):
shelf.resolve_version(index, 'nope')
# --- verified atomic unpack -------------------------------------------------
def test_unpack_verified_places_tree(tmp_path, keypair):
priv, pub = keypair
src = _write_plugin(tmp_path / 'src' / 'demo')
artifact = packaging.pack(src, priv, out_dir=tmp_path / 'dist')
plugins_dir = tmp_path / 'live'
plugins_dir.mkdir()
target = shelf.unpack_verified(artifact, [pub], plugins_dir, 'demo')
assert target == plugins_dir / 'demo'
assert (target / 'manifest.json').exists()
assert (target / signing.PROVENANCE_NAME).exists()
# staging cleaned up
assert not (plugins_dir / shelf.STAGING_DIR / 'demo').exists()
def test_unpack_verified_refuses_tampered(tmp_path, keypair):
priv, pub = keypair
src = _write_plugin(tmp_path / 'src' / 'demo')
artifact = packaging.pack(src, priv, out_dir=tmp_path / 'dist')
# tamper a member, keep provenance + sig
with zipfile.ZipFile(artifact) as zin:
members = {n: zin.read(n) for n in zin.namelist()}
members['manifest.json'] += b'tampered'
with zipfile.ZipFile(artifact, 'w', zipfile.ZIP_DEFLATED) as zout:
for name, data in members.items():
zout.writestr(name, data)
plugins_dir = tmp_path / 'live'
plugins_dir.mkdir()
with pytest.raises(ValueError):
shelf.unpack_verified(artifact, [pub], plugins_dir, 'demo')
# nothing placed
assert not (plugins_dir / 'demo').exists()
def test_unpack_verified_wrong_name_refused(tmp_path, keypair):
priv, pub = keypair
src = _write_plugin(tmp_path / 'src' / 'demo')
artifact = packaging.pack(src, priv, out_dir=tmp_path / 'dist')
plugins_dir = tmp_path / 'live'
plugins_dir.mkdir()
with pytest.raises(ValueError):
shelf.unpack_verified(artifact, [pub], plugins_dir, 'somethingelse')

View File

@@ -0,0 +1,167 @@
"""Verify-at-load / verify-at-migrate enforcement tests (ADR-013 Phase 2).
The security model rests on: with PLUGIN_REQUIRE_SIGNED set, a plugin only
loads or migrates when its tree matches a trusted signature; unsigned, tampered,
or wrong-key plugins are refused; and the dev-trust exemption only relaxes this
under DEBUG/TESTING.
"""
import json
import pytest
from shopdb.exceptions import PluginError
from shopdb.plugins import signing
from shopdb.plugins.loader import PluginLoader
from shopdb.plugins.migrations import PluginMigrationManager
from shopdb.plugins.registry import PluginRegistry
from shopdb.plugins.verification import PluginVerifier
def _write_loadable_plugin(plugins_dir, name='demo'):
"""A minimal plugin whose plugin.py actually imports + instantiates."""
pdir = plugins_dir / name
pdir.mkdir(parents=True, exist_ok=True)
(pdir / 'manifest.json').write_text(json.dumps({
'name': name, 'version': '1.0.0', 'description': 'demo',
'dependencies': [], 'core_version': '>=0.1.0,<1.0.0',
'api_prefix': f'/api/{name}',
}))
(pdir / '__init__.py').write_text('')
(pdir / 'plugin.py').write_text(
'from shopdb.plugins.base import BasePlugin, PluginMeta\n\n'
'class ThePlugin(BasePlugin):\n'
' @property\n'
' def meta(self):\n'
f' return PluginMeta(name={name!r}, version="1.0.0",\n'
' description="demo")\n'
' def get_blueprint(self):\n'
' return None\n'
' def get_models(self):\n'
' return []\n'
)
return pdir
def _stamp(pdir, private_key, name='demo', version='1.0.0'):
provenance = signing.build_provenance(pdir, name, version, 'publisher')
provenance_bytes = signing.serialize_provenance(provenance)
(pdir / signing.PROVENANCE_NAME).write_bytes(provenance_bytes)
(pdir / signing.PROVENANCE_SIG).write_bytes(
signing.sign(private_key, provenance_bytes))
@pytest.fixture
def keypair(tmp_path):
private_pem, public_pem = signing.generate_keypair()
pub_path = tmp_path / 'curator.pub'
pub_path.write_bytes(public_pem)
return signing.load_private_key(private_pem), str(pub_path)
# --- PluginVerifier policy --------------------------------------------------
def test_enforcement_off_allows_unsigned(tmp_path):
_write_loadable_plugin(tmp_path / 'plugins')
verifier = PluginVerifier(tmp_path / 'plugins', require_signed=False)
ok, _ = verifier.check('demo')
assert ok
def test_require_signed_without_keys_refuses_all(tmp_path):
_write_loadable_plugin(tmp_path / 'plugins')
verifier = PluginVerifier(tmp_path / 'plugins', require_signed=True,
trusted_key_paths=[])
ok, reason = verifier.check('demo')
assert not ok
assert 'no trusted keys' in reason
def test_signed_plugin_passes(tmp_path, keypair):
private_key, pub_path = keypair
pdir = _write_loadable_plugin(tmp_path / 'plugins')
_stamp(pdir, private_key)
verifier = PluginVerifier(tmp_path / 'plugins', require_signed=True,
trusted_key_paths=[pub_path])
ok, reason = verifier.check('demo')
assert ok, reason
def test_tampered_plugin_refused(tmp_path, keypair):
private_key, pub_path = keypair
pdir = _write_loadable_plugin(tmp_path / 'plugins')
_stamp(pdir, private_key)
(pdir / 'plugin.py').write_text('# tampered after signing\n')
verifier = PluginVerifier(tmp_path / 'plugins', require_signed=True,
trusted_key_paths=[pub_path])
ok, reason = verifier.check('demo')
assert not ok
assert 'hash mismatch' in reason
def test_wrong_key_refused(tmp_path, keypair):
private_key, _ = keypair
other_pub = tmp_path / 'attacker.pub'
other_pub.write_bytes(signing.generate_keypair()[1])
pdir = _write_loadable_plugin(tmp_path / 'plugins')
_stamp(pdir, private_key)
verifier = PluginVerifier(tmp_path / 'plugins', require_signed=True,
trusted_key_paths=[str(other_pub)])
ok, reason = verifier.check('demo')
assert not ok
assert 'signature does not match' in reason
def test_dev_trust_dir_only_relaxes_in_dev(tmp_path, keypair):
_, pub_path = keypair
_write_loadable_plugin(tmp_path / 'plugins') # unsigned
plugins_dir = tmp_path / 'plugins'
dev = PluginVerifier(plugins_dir, require_signed=True,
trusted_key_paths=[pub_path],
dev_trust_dirs=[str(plugins_dir)], is_dev=True)
ok, reason = dev.check('demo')
assert ok and reason == 'dev-trusted'
prod = PluginVerifier(plugins_dir, require_signed=True,
trusted_key_paths=[pub_path],
dev_trust_dirs=[str(plugins_dir)], is_dev=False)
ok2, _ = prod.check('demo')
assert not ok2
# --- verify-at-load and verify-at-migrate integration -----------------------
def test_loader_refuses_unsigned_when_enforced(tmp_path, app, db):
plugins_dir = tmp_path / 'plugins'
_write_loadable_plugin(plugins_dir)
registry = PluginRegistry(tmp_path / 'plugins.json')
loader = PluginLoader(plugins_dir, registry)
loader.verifier = PluginVerifier(plugins_dir, require_signed=True,
trusted_key_paths=[])
# TESTING app is strict -> the refusal surfaces as a raise.
with pytest.raises(PluginError):
loader.load_plugin('demo', app, db)
def test_loader_loads_signed_when_enforced(tmp_path, app, db, keypair):
private_key, pub_path = keypair
plugins_dir = tmp_path / 'plugins'
pdir = _write_loadable_plugin(plugins_dir)
_stamp(pdir, private_key)
registry = PluginRegistry(tmp_path / 'plugins.json')
loader = PluginLoader(plugins_dir, registry)
loader.verifier = PluginVerifier(plugins_dir, require_signed=True,
trusted_key_paths=[pub_path])
plugin = loader.load_plugin('demo', app, db)
assert plugin is not None
assert plugin.meta.name == 'demo'
def test_migrate_refused_when_unverified(tmp_path):
plugins_dir = tmp_path / 'plugins'
_write_loadable_plugin(plugins_dir)
manager = PluginMigrationManager(plugins_dir, 'sqlite://')
manager.verifier = PluginVerifier(plugins_dir, require_signed=True,
trusted_key_paths=[])
assert manager.run_plugin_migrations('demo') is False