ge-aerospace/shopdb-flask
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ADR-013 Phase 2: enforcement + signed shelf + adopt

Browse Source 
Completes the marketplace security model. Verification stops being advisory:
a plugin only loads or migrates when its tree matches a trusted signature, and
plugins are pulled from a signed shelf with anti-rollback and revocation.

Enforcement (default OFF - existing deploys unchanged):
- verification.py PluginVerifier, shared by the loader (verify-at-load, before
  plugin.py is imported) and the migration manager (verify-at-migrate, before
  any DDL). Fail-closed: an unsigned/tampered/wrong-key plugin does not run.
- Gated by PLUGIN_REQUIRE_SIGNED. PLUGIN_DEV_TRUST_DIRS exempts named dirs but
  only under DEBUG/TESTING; production ignores it.
- flask plugin stamp-bundled writes provenance into in-tree plugins so
  verify-at-load applies to bundled plugins too (image build step).
- tier:core manifest guard: uninstall/disable refuse a core-tier plugin.

Shelf (shelf.py):
- Signed shelf-index.json (+ .sig): monotonic serial (a site refuses an older
  index - anti-rollback), revoked list carried across builds, per-entry
  version/tier/core_version for browse. Index is a browse layer only; adopt
  reads security-bearing fields from the verified artifact.
- flask plugin shelf-build / shelf-list / adopt / audit. adopt verifies index +
  artifact (signature + every file hash), unpacks to staging, re-verifies, then
  atomically moves into place and installs+enables the closure. Refuses a
  downgrade without --force-downgrade. Anti-rollback serial stored in
  instance/shelf-state.json.
- config PLUGIN_SHELF_DIR; the app only reads the folder, never speaks a
  network. .env.example + docs/PLUGIN-SIGNING.md document the flow.

22 tests: verifier policy (off / no-keys / signed / tampered / wrong-key /
dev-exempt), verify-at-load + verify-at-migrate integration, tier guard, index
sign/verify + tamper/wrong-key, serial state, revocation, version resolution,
verified atomic unpack + tamper refusal. Live-smoked keygen->pack->shelf-build
->list->adopt->audit + serial guard. 1050 pass, naming green.
This commit is contained in:
cproudlock
2026-07-18 20:44:54 -04:00
parent 86f5f1be68
commit 5b19f3b554
12 changed files with 1048 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
.env.example
View File

@@ -66,6 +66,19 @@ ZABBIX_TOKEN=
# to verify signed plugin artifacts. Delivered with this config, NEVER from the
# plugin shelf. Empty on a site that does not adopt marketplace plugins.
# PLUGIN_TRUSTED_KEYS=/etc/shopdb/keys/curator.pub
#
# Enforce signatures: a plugin only loads/migrates if its tree matches a
# trusted signature. Default off. Turn on only after stamping plugins
# (flask plugin stamp-bundled) and pinning keys above.
# PLUGIN_REQUIRE_SIGNED=false
#
# Dev-only: directories whose UNSIGNED plugins are trusted, honored ONLY under
# DEBUG/TESTING (external-repo/symlink dev). Production ignores this.
# PLUGIN_DEV_TRUST_DIRS=/home/dev/my-plugin-repo
#
# Read-only folder the app pulls plugin artifacts from (a SharePoint-synced or
# copied shelf). The app reads this folder; it never speaks any network.
# PLUGIN_SHELF_DIR=/srv/shopdb/plugin-shelf
# ---- Employee directory database (optional, read-only) ----
# Separate HR/employee lookup DB consumed by the notifications plugin and the

61
docs/PLUGIN-SIGNING.md
View File

@@ -67,9 +67,60 @@ Keys are read only from this deployed config, never from the shelf - a folder an
attacker could write must not also carry the keys that authenticate it. With no
keys set, `validate` on an artifact fails closed (unverifiable).
## What is NOT in this phase
## Enforce signatures (Phase 2)
Verification is available but not yet enforced automatically at plugin load or
migrate time, and there is no `adopt` command that pulls from a shelf yet. Those
land in Phase 2 (verify-at-load, verify-at-migrate, the signed shelf index).
For now, verify artifacts manually with `validate` before installing.
By default nothing is enforced - plugins load unsigned, as before. To require
signatures on a site:
1. Stamp the plugins the image ships with, so verify-at-load applies to them
too (run at image build with the site/build key):
```
flask plugin stamp-bundled --key ./keys/curator.key
```
This writes `PROVENANCE.json` + `PROVENANCE.sig` into each in-tree plugin.
2. Pin the public key(s) and turn enforcement on (site config):
```
PLUGIN_TRUSTED_KEYS=/etc/shopdb/keys/curator.pub
PLUGIN_REQUIRE_SIGNED=true
```
Now a plugin only loads (verify-at-load) or migrates (verify-at-migrate) when
its tree matches a trusted signature. An unsigned, tampered, or wrong-key
plugin is refused - fail-closed. `PLUGIN_DEV_TRUST_DIRS` exempts named
directories, but ONLY under DEBUG/TESTING (the external-repo dev workflow);
production ignores it.
## The shelf and adopt (Phase 2)
A shelf is a read-only folder of artifacts plus a signed index. The app reads
`PLUGIN_SHELF_DIR`; it never talks to SharePoint - a sync (or robocopy/USB)
populates that folder, and the signature makes the transport untrusted and
interchangeable.
Publish (curator, after packing artifacts into the shelf folder):
```
flask plugin shelf-build --dir /srv/shelf --key ./keys/curator.key --serial 3
```
The index carries a monotonic `serial` (a site refuses an index older than the
last it saw) and a `revoked` list (carried forward across builds). Bump
`--serial` on every publish.
On a site:
```
flask plugin shelf-list # browse (verifies index + serial)
flask plugin adopt printers # or printers==1.2.0
flask plugin audit # warn if an installed version is revoked
```
`adopt` verifies the shelf index and the artifact (signature + every file
hash), unpacks into a staging area, re-verifies, then atomically moves it into
place and installs + enables it with its dependency closure. It refuses a
downgrade unless `--force-downgrade`. Run `flask plugin upgrade-all` and restart
afterward so migrations apply and routes register.

19
shopdb/config.py
View File

@@ -70,6 +70,25 @@ class Config:
if path.strip()
]
# When true, a plugin only loads/migrates if its tree matches a trusted
# signature (verify-at-load, verify-at-migrate). Default false = existing
# behavior. Turn on only after stamping plugins + pinning keys.
PLUGIN_REQUIRE_SIGNED = os.environ.get(
'PLUGIN_REQUIRE_SIGNED', 'false').lower() == 'true'
# Directories whose unsigned plugins are trusted - honored ONLY under
# DEBUG/TESTING (the external-repo/symlink dev workflow). Production ignores.
PLUGIN_DEV_TRUST_DIRS = [
path.strip()
for path in os.environ.get('PLUGIN_DEV_TRUST_DIRS', '').split(os.pathsep)
if path.strip()
]
# Read-only folder the app pulls plugin artifacts from (a SharePoint-synced
# or copied shelf). Empty = no shelf configured. The app never speaks any
# network protocol; it reads this folder.
PLUGIN_SHELF_DIR = os.environ.get('PLUGIN_SHELF_DIR', '')
ZABBIX_ENABLED = os.environ.get('ZABBIX_ENABLED', 'false').lower() == 'true'
ZABBIX_URL = os.environ.get('ZABBIX_URL', '')
ZABBIX_TOKEN = os.environ.get('ZABBIX_TOKEN', '')

34
shopdb/plugins/__init__.py
View File

@@ -39,6 +39,7 @@ class PluginManager:
self.registry: Optional[PluginRegistry] = None
self.loader: Optional[PluginLoader] = None
self.migration_manager: Optional[PluginMigrationManager] = None
self.verifier = None
self._app: Optional[Flask] = None
self._db = None
# API prefixes already claimed by a registered plugin blueprint, to
@@ -66,6 +67,19 @@ class PluginManager:
app.config.get('SQLALCHEMY_DATABASE_URI')
)
# One trust policy, shared by verify-at-load and verify-at-migrate.
from .verification import PluginVerifier
verifier = PluginVerifier(
plugins_dir,
require_signed=app.config.get('PLUGIN_REQUIRE_SIGNED', False),
trusted_key_paths=app.config.get('PLUGIN_TRUSTED_KEYS', []),
dev_trust_dirs=app.config.get('PLUGIN_DEV_TRUST_DIRS', []),
is_dev=bool(app.config.get('DEBUG') or app.config.get('TESTING')),
)
self.verifier = verifier
self.loader.verifier = verifier
self.migration_manager.verifier = verifier
# Load enabled plugins
self._load_enabled_plugins()
@@ -257,6 +271,18 @@ class PluginManager:
"Seeded %d permission(s) for plugin %s",
created, plugin.meta.name)
def _is_core_tier(self, name: str) -> bool:
"""True when the plugin's manifest marks it tier=core (mandatory).
A core-tier plugin refuses uninstall/disable so a site cannot remove a
plugin its deployment depends on. No plugin ships tier=core today; the
guard makes that a manifest edit, not a code change (ADR-013).
"""
try:
return self.loader.load_manifest(name).get('tier') == 'core'
except PluginError:
return False
def _installed_dependents(self, name: str,
enabled_only: bool = False) -> List[str]:
"""Installed plugins that declare `name` as a hard dependency.
@@ -293,6 +319,10 @@ class PluginManager:
logger.warning(f"Plugin {name} is not installed")
return False
if self._is_core_tier(name):
logger.error(f"Cannot uninstall {name}: it is a core-tier plugin")
return False
# Refuse if any INSTALLED plugin (enabled or not) depends on this one.
dependents = self._installed_dependents(name)
if dependents:
@@ -372,6 +402,10 @@ class PluginManager:
logger.info(f"Plugin {name} is already disabled")
return True
if self._is_core_tier(name):
logger.error(f"Cannot disable {name}: it is a core-tier plugin")
return False
# Refuse if any ENABLED plugin depends on this one (a disabled dependent
# is not running, so it does not block). Manifests read from disk.
dependents = self._installed_dependents(name, enabled_only=True)

277
shopdb/plugins/cli.py
View File

@@ -553,6 +553,283 @@ def apply_profile(profile: str):
"restart so new blueprints/routes register.", fg='yellow'))
def _shelf_keys(pubkeys):
"""Trusted keys for shelf/artifact verification: explicit --pubkey first,
else the site's PLUGIN_TRUSTED_KEYS."""
from .signing import load_trusted_keys
paths = list(pubkeys) or current_app.config.get('PLUGIN_TRUSTED_KEYS', [])
return load_trusted_keys(paths), paths
@plugin_cli.command('stamp-bundled')
@click.option('--key', 'key_path', required=True, type=click.Path(exists=True),
help='Private signing key PEM')
@click.option('--publisher', default='', help='Publisher id in the provenance')
@click.argument('name', required=False)
@with_appcontext
def stamp_bundled(key_path, publisher, name):
"""Write PROVENANCE.json/.sig into in-tree plugin dirs (image build step).
Stamps one plugin (NAME) or every discovered plugin, so verify-at-load
applies to bundled plugins identically to adopted ones. Run at image build
with the site/build key, then set PLUGIN_REQUIRE_SIGNED.
"""
from .signing import (load_private_key, build_provenance,
serialize_provenance, sign,
PROVENANCE_NAME, PROVENANCE_SIG)
pm = current_app.extensions.get('plugin_manager')
if not pm:
click.echo(click.style("Plugin manager not initialized", fg='red'))
raise SystemExit(1)
private_key = load_private_key(Path(key_path).read_bytes())
names = [name] if name else pm.loader.discover_plugins()
plugins_dir = Path(pm.loader.plugins_dir)
for plugin_name in names:
plugin_dir = plugins_dir / plugin_name
manifest = pm.loader.load_manifest(plugin_name)
provenance = build_provenance(
plugin_dir, manifest['name'], manifest['version'], publisher)
provenance_bytes = serialize_provenance(provenance)
(plugin_dir / PROVENANCE_NAME).write_bytes(provenance_bytes)
(plugin_dir / PROVENANCE_SIG).write_bytes(
sign(private_key, provenance_bytes))
click.echo(click.style(f" stamped {plugin_name}", fg='green'))
@plugin_cli.command('shelf-build')
@click.option('--dir', 'shelf_dir', required=True,
type=click.Path(exists=True, file_okay=False),
help='Shelf directory holding the artifacts')
@click.option('--key', 'key_path', required=True,
type=click.Path(exists=True), help='Private signing key PEM')
@click.option('--serial', type=int, required=True,
help='Index serial (must exceed the current published serial)')
@click.option('--publisher', default='')
def shelf_build(shelf_dir, key_path, serial, publisher):
"""Build + sign the shelf index from the artifacts in DIR.
Carries forward the existing revoked list and refuses a serial that does not
advance past the published one (a lower serial would let a site's
anti-rollback reject the new index).
"""
import json as _json
from .signing import load_private_key
from .shelf import build_index, INDEX_NAME
shelf = Path(shelf_dir)
revoked = []
existing = shelf / INDEX_NAME
if existing.exists():
try:
prev = _json.loads(existing.read_text())
revoked = prev.get('revoked', [])
if serial <= int(prev.get('serial', -1)):
click.echo(click.style(
f"serial {serial} must exceed current "
f"{prev.get('serial')}", fg='red'))
raise SystemExit(1)
except (ValueError, TypeError):
pass
private_key = load_private_key(Path(key_path).read_bytes())
index = build_index(shelf, private_key, serial,
publisher=publisher, revoked=revoked)
total = sum(len(v) for v in index['plugins'].values())
click.echo(click.style(
f"Shelf index signed: serial {serial}, {total} artifact(s), "
f"{len(revoked)} revoked.", fg='green'))
@plugin_cli.command('shelf-list')
@click.option('--pubkey', 'pubkeys', multiple=True, type=click.Path(exists=True))
@with_appcontext
def shelf_list(pubkeys):
"""Browse the configured shelf (verifies index signature + serial)."""
from shopdb import __contract_version__
from packaging.specifiers import SpecifierSet
from packaging.version import Version
from .shelf import load_index, read_last_serial, write_last_serial, is_revoked
shelf_dir = current_app.config.get('PLUGIN_SHELF_DIR')
if not shelf_dir:
click.echo(click.style("PLUGIN_SHELF_DIR not configured", fg='red'))
raise SystemExit(1)
keys, _ = _shelf_keys(pubkeys)
index, errors = load_index(shelf_dir, keys)
for err in errors:
click.echo(click.style(f" {err}", fg='red'))
if index is None:
raise SystemExit(1)
last = read_last_serial(current_app.instance_path)
if index['serial'] < last:
click.echo(click.style(
f"REFUSING shelf: index serial {index['serial']} < last seen "
f"{last} (rollback?)", fg='red'))
raise SystemExit(1)
if errors:
raise SystemExit(1)
write_last_serial(current_app.instance_path, index['serial'])
click.echo("")
click.echo(click.style(
f"Shelf (serial {index['serial']}):", fg='cyan', bold=True))
for name in sorted(index.get('plugins', {})):
for entry in index['plugins'][name]:
compat = 'ok'
spec = entry.get('core_version', '')
if spec and Version(__contract_version__) not in SpecifierSet(spec):
compat = f'needs core {spec}'
flags = []
if is_revoked(index, name, entry['version']):
flags.append('REVOKED')
if compat != 'ok':
flags.append(compat)
tag = (' [' + ', '.join(flags) + ']') if flags else ''
click.echo(f" {name:20} {entry['version']:10} "
f"({entry.get('tier', 'optional')}){tag}")
@plugin_cli.command('adopt')
@click.argument('spec')
@click.option('--pubkey', 'pubkeys', multiple=True, type=click.Path(exists=True))
@click.option('--force-downgrade', is_flag=True,
help='Allow adopting a version lower than the installed one')
@with_appcontext
def adopt_plugin(spec, pubkeys, force_downgrade):
"""Pull a plugin from the shelf, verify it, and install + enable it.
SPEC is a plugin name or name==version. Verifies the shelf index and the
artifact signature + hashes, unpacks atomically only after verification,
then installs + enables the plugin and its dependency closure.
Usage: flask plugin adopt printers (or printers==1.2.0)
"""
from packaging.version import Version
from ..exceptions import PluginError
from .shelf import (load_index, read_last_serial, write_last_serial,
resolve_version, unpack_verified)
pm = current_app.extensions.get('plugin_manager')
if not pm:
click.echo(click.style("Plugin manager not initialized", fg='red'))
raise SystemExit(1)
name, _, version = spec.partition('==')
name = name.strip()
version = version.strip() or None
shelf_dir = current_app.config.get('PLUGIN_SHELF_DIR')
if not shelf_dir:
click.echo(click.style("PLUGIN_SHELF_DIR not configured", fg='red'))
raise SystemExit(1)
keys, _ = _shelf_keys(pubkeys)
index, errors = load_index(shelf_dir, keys)
for err in errors:
click.echo(click.style(f" {err}", fg='red'))
if index is None or errors:
raise SystemExit(1)
last = read_last_serial(current_app.instance_path)
if index['serial'] < last:
click.echo(click.style(
f"REFUSING shelf: serial {index['serial']} < last seen {last} "
f"(rollback?)", fg='red'))
raise SystemExit(1)
try:
resolved_version, artifact_name = resolve_version(index, name, version)
except (KeyError, ValueError) as e:
click.echo(click.style(f"{e}", fg='red'))
raise SystemExit(1)
state = pm.registry.get(name)
if state and not force_downgrade:
try:
if Version(resolved_version) < Version(state.version):
click.echo(click.style(
f"Refusing downgrade {state.version} -> {resolved_version} "
f"(use --force-downgrade)", fg='red'))
raise SystemExit(1)
except Exception:
pass
artifact_path = Path(shelf_dir) / artifact_name
plugins_dir = Path(pm.loader.plugins_dir)
click.echo(f"Verifying + unpacking {name} {resolved_version} ...")
try:
unpack_verified(artifact_path, keys, plugins_dir, name)
except ValueError as e:
click.echo(click.style(f" verification failed: {e}", fg='red'))
raise SystemExit(1)
# Clear any cached manifest so the freshly unpacked one is read.
pm.loader._manifests.pop(name, None)
try:
result = pm.apply_profile([name])
except PluginError as e:
click.echo(click.style(f" {e}", fg='red'))
click.echo(click.style(
" (adopt the missing dependency plugins first)", fg='yellow'))
raise SystemExit(1)
write_last_serial(current_app.instance_path, index['serial'])
click.echo("")
click.echo(click.style(
f"Adopted {name} {resolved_version}.", fg='green'))
if result['installed']:
click.echo(f" installed: {', '.join(result['installed'])}")
if result['enabled']:
click.echo(f" enabled: {', '.join(result['enabled'])}")
click.echo(click.style(
"Run 'flask plugin upgrade-all' then restart so routes register.",
fg='yellow'))
@plugin_cli.command('audit')
@click.option('--pubkey', 'pubkeys', multiple=True, type=click.Path(exists=True))
@with_appcontext
def audit_plugins(pubkeys):
"""Warn about installed plugins whose version is revoked on the shelf."""
from .shelf import load_index, is_revoked
pm = current_app.extensions.get('plugin_manager')
if not pm:
click.echo(click.style("Plugin manager not initialized", fg='red'))
raise SystemExit(1)
shelf_dir = current_app.config.get('PLUGIN_SHELF_DIR')
if not shelf_dir:
click.echo(click.style("PLUGIN_SHELF_DIR not configured", fg='red'))
raise SystemExit(1)
keys, _ = _shelf_keys(pubkeys)
index, errors = load_index(shelf_dir, keys)
for err in errors:
click.echo(click.style(f" {err}", fg='red'))
if index is None:
raise SystemExit(1)
flagged = []
for name, state in pm.registry.get_all().items():
if is_revoked(index, name, state.version):
flagged.append(f"{name} {state.version}")
click.echo("")
if flagged:
click.echo(click.style(
"REVOKED plugins installed:", fg='red', bold=True))
for item in flagged:
click.echo(click.style(f" {item}", fg='red'))
raise SystemExit(1)
click.echo(click.style("No installed plugin is revoked.", fg='green'))
@plugin_cli.command('migrate')
@click.argument('name')
@click.option('--revision', default='head', help='Target revision')

13
shopdb/plugins/loader.py
View File

@@ -45,6 +45,8 @@ class PluginLoader:
self._loaded_plugins: Dict[str, BasePlugin] = {}
self._plugin_classes: Dict[str, Type[BasePlugin]] = {}
self._manifests: Dict[str, dict] = {}
# Set by PluginManager.init_app; a PluginVerifier or None (no policy).
self.verifier = None
def _is_strict_mode(self, app: Optional[Flask]) -> bool:
"""Return True if loader should re-raise instead of isolating failures."""
@@ -198,6 +200,17 @@ class PluginLoader:
manifest = self.load_manifest(name)
self.check_contract_version(name, __contract_version__)
# verify-at-load: refuse to import plugin.py unless the tree matches
# a trusted signature (when the site enforces it). Runs BEFORE any
# plugin code is imported, so a tampered/unsigned plugin never runs.
if self.verifier is not None:
ok, reason = self.verifier.check(name)
if not ok:
raise PluginError(
f'Plugin {name} failed signature verification: {reason}',
plugin_name=name,
)
for dep in manifest.get('dependencies', []):
if not self.registry.is_enabled(dep):
raise PluginDependencyError(

12
shopdb/plugins/migrations.py
View File

@@ -18,6 +18,8 @@ class PluginMigrationManager:
def __init__(self, plugins_dir: Path, database_url: str):
self.plugins_dir = plugins_dir
self.database_url = database_url
# Set by PluginManager.init_app; a PluginVerifier or None (no policy).
self.verifier = None
def get_migrations_dir(self, plugin_name: str) -> Optional[Path]:
"""Get migrations directory for a plugin."""
@@ -36,6 +38,16 @@ class PluginMigrationManager:
Uses flask db upgrade with the plugin's migrations directory.
"""
# verify-at-migrate: never run a plugin's DDL (full DB rights) from an
# unverified tree when the site enforces signing.
if self.verifier is not None:
ok, reason = self.verifier.check(plugin_name)
if not ok:
logger.error(
"Refusing migrations for %s: signature verification failed "
"(%s)", plugin_name, reason)
return False
migrations_dir = self.get_migrations_dir(plugin_name)
if not migrations_dir:

185
shopdb/plugins/shelf.py Normal file
View File

@@ -0,0 +1,185 @@
"""Signed plugin shelf: index build/verify, anti-rollback state, adopt mechanics
(ADR-013 Phase 2).
The shelf is a read-only folder of .shopdbplugin artifacts plus a signed
shelf-index.json (+ .sig). The index carries a monotonic serial (a site refuses
an index older than the last it saw) and a revoked list. Browse reads the index;
ADOPT always re-verifies the artifact's own signed manifest, never trusting the
index for anything security-bearing.
"""
import json
import shutil
import zipfile
from pathlib import Path
from packaging.version import Version, InvalidVersion
from . import signing, packaging
INDEX_NAME = 'shelf-index.json'
INDEX_SIG = 'shelf-index.sig'
STATE_NAME = 'shelf-state.json'
STAGING_DIR = '.staging'
def _read_artifact_manifest(artifact_path) -> dict:
with zipfile.ZipFile(artifact_path) as archive:
return json.loads(archive.read('manifest.json'))
def build_index(shelf_dir, private_key, serial: int,
publisher: str = '', revoked=None) -> dict:
"""Scan the shelf for artifacts, build and SIGN the index. Caller supplies a
serial that must exceed any previously published one."""
shelf_dir = Path(shelf_dir)
plugins = {}
for artifact in sorted(shelf_dir.glob(f'*{packaging.ARTIFACT_SUFFIX}')):
manifest = _read_artifact_manifest(artifact)
plugins.setdefault(manifest['name'], []).append({
'version': manifest['version'],
'tier': manifest.get('tier', 'optional'),
'core_version': manifest.get('core_version', ''),
'artifact': artifact.name,
})
index = {
'serial': int(serial),
'publisher': publisher or '',
'plugins': plugins,
'revoked': list(revoked or []),
}
index_bytes = signing.serialize_provenance(index) # canonical, sorted bytes
(shelf_dir / INDEX_NAME).write_bytes(index_bytes)
(shelf_dir / INDEX_SIG).write_bytes(signing.sign(private_key, index_bytes))
return index
def load_index(shelf_dir, public_keys):
"""(index_or_None, errors). Verifies the index signature. Serial-vs-state is
the caller's check (needs the instance path)."""
shelf_dir = Path(shelf_dir)
index_path = shelf_dir / INDEX_NAME
sig_path = shelf_dir / INDEX_SIG
if not index_path.exists() or not sig_path.exists():
return None, ['shelf has no signed index (shelf-index.json/.sig)']
index_bytes = index_path.read_bytes()
signature = sig_path.read_bytes()
errors = []
if not public_keys:
errors.append('no trusted keys - shelf index unverifiable')
elif not signing.verify(public_keys, index_bytes, signature):
errors.append('shelf index signature does not match any trusted key')
try:
index = json.loads(index_bytes)
except json.JSONDecodeError as exc:
return None, errors + [f'shelf index is not valid JSON: {exc}']
return index, errors
def read_last_serial(instance_path) -> int:
state = Path(instance_path) / STATE_NAME
if state.exists():
try:
return int(json.loads(state.read_text()).get('last_serial', -1))
except (json.JSONDecodeError, ValueError, TypeError):
return -1
return -1
def write_last_serial(instance_path, serial: int) -> None:
state = Path(instance_path) / STATE_NAME
state.parent.mkdir(parents=True, exist_ok=True)
state.write_text(json.dumps({'last_serial': int(serial)}, indent=2))
def is_revoked(index: dict, name: str, version: str) -> bool:
for entry in index.get('revoked', []):
if entry.get('name') == name and entry.get('version') == version:
return True
return False
def _safe_version(value):
try:
return Version(value)
except InvalidVersion:
return None
def resolve_version(index: dict, name: str, version: str = None):
"""Pick the artifact filename for name[==version] from the index.
With no version, picks the highest non-revoked version. Returns
(version, artifact_filename) or raises KeyError/ValueError.
"""
candidates = index.get('plugins', {}).get(name)
if not candidates:
raise KeyError(f'plugin {name} not on the shelf')
if version is not None:
for entry in candidates:
if entry['version'] == version:
if is_revoked(index, name, version):
raise ValueError(f'{name} {version} is revoked')
return version, entry['artifact']
raise KeyError(f'{name} {version} not on the shelf')
live = [e for e in candidates if not is_revoked(index, name, e['version'])]
if not live:
raise ValueError(f'every published version of {name} is revoked')
best = max(live, key=lambda e: (_safe_version(e['version']) or Version('0'),))
return best['version'], best['artifact']
def unpack_verified(artifact_path, public_keys, plugins_dir, name: str):
"""Verify an artifact, then atomically place it at plugins_dir/name.
Verifies signature + every file hash in a staging area BEFORE it can be
imported; only a fully verified tree is moved into place. Returns the final
plugin directory. Raises ValueError with all problems on any failure.
"""
plugins_dir = Path(plugins_dir)
manifest, errors = packaging.verify_artifact(artifact_path, public_keys)
if errors:
raise ValueError('; '.join(errors))
if manifest.get('name') != name:
raise ValueError(
f"artifact manifest name '{manifest.get('name')}' != '{name}'")
staging = plugins_dir / STAGING_DIR
staging.mkdir(parents=True, exist_ok=True)
staged = staging / name
if staged.exists():
shutil.rmtree(staged)
with zipfile.ZipFile(artifact_path) as archive:
archive.extractall(staged)
# Re-verify the unpacked tree before it goes live (defense against an
# extract that did not land what we verified).
_, staged_errors = packaging.verify_dir(staged, public_keys)
if staged_errors:
shutil.rmtree(staged, ignore_errors=True)
raise ValueError('staged tree failed re-verification: '
+ '; '.join(staged_errors))
target = plugins_dir / name
backup = staging / f'{name}.backup'
if target.exists():
if backup.exists():
shutil.rmtree(backup)
target.rename(backup)
try:
staged.rename(target)
except OSError:
# cross-device or rename race: fall back to copy, keep backup on failure
shutil.copytree(staged, target)
shutil.rmtree(staged, ignore_errors=True)
if backup.exists():
shutil.rmtree(backup, ignore_errors=True)
return target

64
shopdb/plugins/verification.py Normal file
View File

@@ -0,0 +1,64 @@
"""Plugin signature enforcement policy (ADR-013 Phase 2).
One PluginVerifier is built per app from config and shared by the loader
(verify-at-load) and the migration manager (verify-at-migrate). A plugin only
loads or migrates when it verifies against a trusted key - so signing stops
being advisory and becomes enforced everywhere plugin code executes.
Default is OFF (PLUGIN_REQUIRE_SIGNED unset), so existing deploys are
unchanged. A site opts into hardening by stamping its plugins
(`flask plugin stamp-bundled`), pinning keys (PLUGIN_TRUSTED_KEYS), and setting
PLUGIN_REQUIRE_SIGNED. Fail-closed: an unverifiable plugin does NOT run.
"""
import logging
from pathlib import Path
from . import signing, packaging
logger = logging.getLogger(__name__)
class PluginVerifier:
"""Decides whether a plugin may load/migrate under the site's trust policy."""
def __init__(self, plugins_dir, require_signed=False, trusted_key_paths=None,
dev_trust_dirs=None, is_dev=False):
self.plugins_dir = Path(plugins_dir)
self.require_signed = bool(require_signed)
self.is_dev = bool(is_dev)
# dev trust dirs only mean anything in dev/test (ADR-013); prod ignores.
self.dev_trust_dirs = [Path(p).resolve() for p in (dev_trust_dirs or [])]
self._keys = signing.load_trusted_keys(trusted_key_paths or [])
def _dev_exempt(self, plugin_dir: Path) -> bool:
# unsigned dev/external-repo plugins pass, but ONLY under DEBUG/TESTING
if not self.is_dev or not self.dev_trust_dirs:
return False
resolved = Path(plugin_dir).resolve()
for trusted in self.dev_trust_dirs:
try:
resolved.relative_to(trusted)
return True
except ValueError:
continue
return False
def check(self, plugin_name: str):
"""(ok, reason). ok True = plugin may load/migrate."""
if not self.require_signed:
return True, 'enforcement off'
plugin_dir = self.plugins_dir / plugin_name
if self._dev_exempt(plugin_dir):
return True, 'dev-trusted'
if not self._keys:
# require-signed with nothing to verify against = refuse everything.
return False, ('PLUGIN_REQUIRE_SIGNED is set but no trusted keys '
'are configured')
_, errors = packaging.verify_dir(plugin_dir, self._keys)
if errors:
return False, '; '.join(errors)
return True, 'verified'

31
tests/test_plugin_lifecycle.py
View File

@@ -58,18 +58,21 @@ def test_registry_migrates_equipment_to_machines(tmp_path):
# --- Manager enable/disable dependency guards + permission seeding -----------
def _write_plugin(plugins_dir, name, deps=None, permissions=None):
def _write_plugin(plugins_dir, name, deps=None, permissions=None, tier=None):
"""Write a minimal valid synthetic plugin (manifest.json + plugin.py)."""
pdir = plugins_dir / name
pdir.mkdir()
(pdir / 'manifest.json').write_text(json.dumps({
manifest = {
'name': name,
'version': '1.0.0',
'description': f'synthetic {name}',
'dependencies': deps or [],
'core_version': '>=0.1.0,<1.0.0',
'api_prefix': f'/api/{name}',
}))
}
if tier:
manifest['tier'] = tier
(pdir / 'manifest.json').write_text(json.dumps(manifest))
(pdir / '__init__.py').write_text('')
(pdir / 'plugin.py').write_text(
'from shopdb.plugins.base import BasePlugin, PluginMeta\n\n'
@@ -286,6 +289,28 @@ def test_validate_bundled_plugin_passes(app):
assert 'valid' in result.output
def test_core_tier_plugin_cannot_be_disabled_or_uninstalled(app, db, tmp_path):
"""A tier=core plugin refuses disable and uninstall (ADR-013)."""
plugins_dir = tmp_path / 'plugins'
plugins_dir.mkdir()
_write_plugin(plugins_dir, 'kernel', tier='core')
registry = PluginRegistry(tmp_path / 'plugins.json')
registry.register('kernel', '1.0.0', enabled=True)
pm = PluginManager()
pm._app = app
pm._db = db
pm.registry = registry
pm.loader = PluginLoader(plugins_dir, registry)
pm.migration_manager = None
assert pm.disable_plugin('kernel') is False
assert pm.registry.is_enabled('kernel')
assert pm.uninstall_plugin('kernel') is False
assert pm.registry.is_installed('kernel')
# --- Seed idempotency -------------------------------------------------------
def test_seed_settings_is_idempotent(app, db):

180
tests/test_plugin_shelf.py Normal file
View File

@@ -0,0 +1,180 @@
"""Signed shelf index + adopt-mechanics tests (ADR-013 Phase 2).
Covers: a signed index round-trips and a tampered/wrong-key index is rejected;
serial anti-rollback state; revocation; version resolution; and unpack_verified
placing only a fully verified tree (refusing a tampered artifact).
"""
import json
import zipfile
import pytest
from shopdb.plugins import signing, packaging, shelf
def _write_plugin(plugin_dir, name='demo', version='1.0.0', tier=None):
plugin_dir.mkdir(parents=True, exist_ok=True)
manifest = {
'name': name, 'version': version, 'description': 'demo',
'dependencies': [], 'core_version': '>=0.1.0,<1.0.0',
'api_prefix': f'/api/{name}',
}
if tier:
manifest['tier'] = tier
(plugin_dir / 'manifest.json').write_text(json.dumps(manifest))
(plugin_dir / 'plugin.py').write_text('# demo\n')
return plugin_dir
@pytest.fixture
def keypair(tmp_path):
private_pem, public_pem = signing.generate_keypair()
return (signing.load_private_key(private_pem),
signing.load_public_key(public_pem))
def _build_shelf(tmp_path, private_key, specs, serial=1, revoked=None):
"""specs: list of (name, version). Packs each into a shelf dir + index."""
shelf_dir = tmp_path / 'shelf'
shelf_dir.mkdir(parents=True, exist_ok=True)
for name, version in specs:
src = tmp_path / 'src' / f'{name}-{version}'
_write_plugin(src, name=name, version=version)
packaging.pack(src, private_key, out_dir=shelf_dir)
shelf.build_index(shelf_dir, private_key, serial, revoked=revoked)
return shelf_dir
# --- index sign/verify ------------------------------------------------------
def test_index_round_trip(tmp_path, keypair):
priv, pub = keypair
shelf_dir = _build_shelf(tmp_path, priv, [('demo', '1.0.0')])
index, errors = shelf.load_index(shelf_dir, [pub])
assert errors == []
assert index['serial'] == 1
assert 'demo' in index['plugins']
def test_tampered_index_rejected(tmp_path, keypair):
priv, pub = keypair
shelf_dir = _build_shelf(tmp_path, priv, [('demo', '1.0.0')])
# change the index bytes but keep the old signature
path = shelf_dir / shelf.INDEX_NAME
data = json.loads(path.read_text())
data['serial'] = 999
path.write_bytes(signing.serialize_provenance(data))
_, errors = shelf.load_index(shelf_dir, [pub])
assert any('signature does not match' in e for e in errors)
def test_index_wrong_key_rejected(tmp_path, keypair):
priv, _ = keypair
shelf_dir = _build_shelf(tmp_path, priv, [('demo', '1.0.0')])
wrong_pub = signing.load_public_key(signing.generate_keypair()[1])
_, errors = shelf.load_index(shelf_dir, [wrong_pub])
assert any('signature does not match' in e for e in errors)
def test_index_no_keys_unverifiable(tmp_path, keypair):
priv, _ = keypair
shelf_dir = _build_shelf(tmp_path, priv, [('demo', '1.0.0')])
_, errors = shelf.load_index(shelf_dir, [])
assert any('unverifiable' in e for e in errors)
# --- anti-rollback state ----------------------------------------------------
def test_serial_state_round_trip(tmp_path):
assert shelf.read_last_serial(tmp_path) == -1
shelf.write_last_serial(tmp_path, 5)
assert shelf.read_last_serial(tmp_path) == 5
# --- revocation + version resolution ----------------------------------------
def test_resolve_highest_non_revoked(tmp_path, keypair):
priv, pub = keypair
shelf_dir = _build_shelf(
tmp_path, priv, [('demo', '1.0.0'), ('demo', '1.2.0')])
index, _ = shelf.load_index(shelf_dir, [pub])
version, artifact = shelf.resolve_version(index, 'demo')
assert version == '1.2.0'
assert artifact == 'demo-1.2.0.shopdbplugin'
def test_resolve_skips_revoked(tmp_path, keypair):
priv, pub = keypair
shelf_dir = _build_shelf(
tmp_path, priv, [('demo', '1.0.0'), ('demo', '1.2.0')],
revoked=[{'name': 'demo', 'version': '1.2.0'}])
index, _ = shelf.load_index(shelf_dir, [pub])
version, _ = shelf.resolve_version(index, 'demo')
assert version == '1.0.0'
def test_resolve_explicit_revoked_raises(tmp_path, keypair):
priv, pub = keypair
shelf_dir = _build_shelf(
tmp_path, priv, [('demo', '1.0.0')],
revoked=[{'name': 'demo', 'version': '1.0.0'}])
index, _ = shelf.load_index(shelf_dir, [pub])
with pytest.raises(ValueError):
shelf.resolve_version(index, 'demo', '1.0.0')
def test_resolve_unknown_raises(tmp_path, keypair):
priv, pub = keypair
shelf_dir = _build_shelf(tmp_path, priv, [('demo', '1.0.0')])
index, _ = shelf.load_index(shelf_dir, [pub])
with pytest.raises(KeyError):
shelf.resolve_version(index, 'nope')
# --- verified atomic unpack -------------------------------------------------
def test_unpack_verified_places_tree(tmp_path, keypair):
priv, pub = keypair
src = _write_plugin(tmp_path / 'src' / 'demo')
artifact = packaging.pack(src, priv, out_dir=tmp_path / 'dist')
plugins_dir = tmp_path / 'live'
plugins_dir.mkdir()
target = shelf.unpack_verified(artifact, [pub], plugins_dir, 'demo')
assert target == plugins_dir / 'demo'
assert (target / 'manifest.json').exists()
assert (target / signing.PROVENANCE_NAME).exists()
# staging cleaned up
assert not (plugins_dir / shelf.STAGING_DIR / 'demo').exists()
def test_unpack_verified_refuses_tampered(tmp_path, keypair):
priv, pub = keypair
src = _write_plugin(tmp_path / 'src' / 'demo')
artifact = packaging.pack(src, priv, out_dir=tmp_path / 'dist')
# tamper a member, keep provenance + sig
with zipfile.ZipFile(artifact) as zin:
members = {n: zin.read(n) for n in zin.namelist()}
members['manifest.json'] += b'tampered'
with zipfile.ZipFile(artifact, 'w', zipfile.ZIP_DEFLATED) as zout:
for name, data in members.items():
zout.writestr(name, data)
plugins_dir = tmp_path / 'live'
plugins_dir.mkdir()
with pytest.raises(ValueError):
shelf.unpack_verified(artifact, [pub], plugins_dir, 'demo')
# nothing placed
assert not (plugins_dir / 'demo').exists()
def test_unpack_verified_wrong_name_refused(tmp_path, keypair):
priv, pub = keypair
src = _write_plugin(tmp_path / 'src' / 'demo')
artifact = packaging.pack(src, priv, out_dir=tmp_path / 'dist')
plugins_dir = tmp_path / 'live'
plugins_dir.mkdir()
with pytest.raises(ValueError):
shelf.unpack_verified(artifact, [pub], plugins_dir, 'somethingelse')

167
tests/test_plugin_verification.py Normal file
View File

@@ -0,0 +1,167 @@
"""Verify-at-load / verify-at-migrate enforcement tests (ADR-013 Phase 2).
The security model rests on: with PLUGIN_REQUIRE_SIGNED set, a plugin only
loads or migrates when its tree matches a trusted signature; unsigned, tampered,
or wrong-key plugins are refused; and the dev-trust exemption only relaxes this
under DEBUG/TESTING.
"""
import json
import pytest
from shopdb.exceptions import PluginError
from shopdb.plugins import signing
from shopdb.plugins.loader import PluginLoader
from shopdb.plugins.migrations import PluginMigrationManager
from shopdb.plugins.registry import PluginRegistry
from shopdb.plugins.verification import PluginVerifier
def _write_loadable_plugin(plugins_dir, name='demo'):
"""A minimal plugin whose plugin.py actually imports + instantiates."""
pdir = plugins_dir / name
pdir.mkdir(parents=True, exist_ok=True)
(pdir / 'manifest.json').write_text(json.dumps({
'name': name, 'version': '1.0.0', 'description': 'demo',
'dependencies': [], 'core_version': '>=0.1.0,<1.0.0',
'api_prefix': f'/api/{name}',
}))
(pdir / '__init__.py').write_text('')
(pdir / 'plugin.py').write_text(
'from shopdb.plugins.base import BasePlugin, PluginMeta\n\n'
'class ThePlugin(BasePlugin):\n'
' @property\n'
' def meta(self):\n'
f' return PluginMeta(name={name!r}, version="1.0.0",\n'
' description="demo")\n'
' def get_blueprint(self):\n'
' return None\n'
' def get_models(self):\n'
' return []\n'
)
return pdir
def _stamp(pdir, private_key, name='demo', version='1.0.0'):
provenance = signing.build_provenance(pdir, name, version, 'publisher')
provenance_bytes = signing.serialize_provenance(provenance)
(pdir / signing.PROVENANCE_NAME).write_bytes(provenance_bytes)
(pdir / signing.PROVENANCE_SIG).write_bytes(
signing.sign(private_key, provenance_bytes))
@pytest.fixture
def keypair(tmp_path):
private_pem, public_pem = signing.generate_keypair()
pub_path = tmp_path / 'curator.pub'
pub_path.write_bytes(public_pem)
return signing.load_private_key(private_pem), str(pub_path)
# --- PluginVerifier policy --------------------------------------------------
def test_enforcement_off_allows_unsigned(tmp_path):
_write_loadable_plugin(tmp_path / 'plugins')
verifier = PluginVerifier(tmp_path / 'plugins', require_signed=False)
ok, _ = verifier.check('demo')
assert ok
def test_require_signed_without_keys_refuses_all(tmp_path):
_write_loadable_plugin(tmp_path / 'plugins')
verifier = PluginVerifier(tmp_path / 'plugins', require_signed=True,
trusted_key_paths=[])
ok, reason = verifier.check('demo')
assert not ok
assert 'no trusted keys' in reason
def test_signed_plugin_passes(tmp_path, keypair):
private_key, pub_path = keypair
pdir = _write_loadable_plugin(tmp_path / 'plugins')
_stamp(pdir, private_key)
verifier = PluginVerifier(tmp_path / 'plugins', require_signed=True,
trusted_key_paths=[pub_path])
ok, reason = verifier.check('demo')
assert ok, reason
def test_tampered_plugin_refused(tmp_path, keypair):
private_key, pub_path = keypair
pdir = _write_loadable_plugin(tmp_path / 'plugins')
_stamp(pdir, private_key)
(pdir / 'plugin.py').write_text('# tampered after signing\n')
verifier = PluginVerifier(tmp_path / 'plugins', require_signed=True,
trusted_key_paths=[pub_path])
ok, reason = verifier.check('demo')
assert not ok
assert 'hash mismatch' in reason
def test_wrong_key_refused(tmp_path, keypair):
private_key, _ = keypair
other_pub = tmp_path / 'attacker.pub'
other_pub.write_bytes(signing.generate_keypair()[1])
pdir = _write_loadable_plugin(tmp_path / 'plugins')
_stamp(pdir, private_key)
verifier = PluginVerifier(tmp_path / 'plugins', require_signed=True,
trusted_key_paths=[str(other_pub)])
ok, reason = verifier.check('demo')
assert not ok
assert 'signature does not match' in reason
def test_dev_trust_dir_only_relaxes_in_dev(tmp_path, keypair):
_, pub_path = keypair
_write_loadable_plugin(tmp_path / 'plugins') # unsigned
plugins_dir = tmp_path / 'plugins'
dev = PluginVerifier(plugins_dir, require_signed=True,
trusted_key_paths=[pub_path],
dev_trust_dirs=[str(plugins_dir)], is_dev=True)
ok, reason = dev.check('demo')
assert ok and reason == 'dev-trusted'
prod = PluginVerifier(plugins_dir, require_signed=True,
trusted_key_paths=[pub_path],
dev_trust_dirs=[str(plugins_dir)], is_dev=False)
ok2, _ = prod.check('demo')
assert not ok2
# --- verify-at-load and verify-at-migrate integration -----------------------
def test_loader_refuses_unsigned_when_enforced(tmp_path, app, db):
plugins_dir = tmp_path / 'plugins'
_write_loadable_plugin(plugins_dir)
registry = PluginRegistry(tmp_path / 'plugins.json')
loader = PluginLoader(plugins_dir, registry)
loader.verifier = PluginVerifier(plugins_dir, require_signed=True,
trusted_key_paths=[])
# TESTING app is strict -> the refusal surfaces as a raise.
with pytest.raises(PluginError):
loader.load_plugin('demo', app, db)
def test_loader_loads_signed_when_enforced(tmp_path, app, db, keypair):
private_key, pub_path = keypair
plugins_dir = tmp_path / 'plugins'
pdir = _write_loadable_plugin(plugins_dir)
_stamp(pdir, private_key)
registry = PluginRegistry(tmp_path / 'plugins.json')
loader = PluginLoader(plugins_dir, registry)
loader.verifier = PluginVerifier(plugins_dir, require_signed=True,
trusted_key_paths=[pub_path])
plugin = loader.load_plugin('demo', app, db)
assert plugin is not None
assert plugin.meta.name == 'demo'
def test_migrate_refused_when_unverified(tmp_path):
plugins_dir = tmp_path / 'plugins'
_write_loadable_plugin(plugins_dir)
manager = PluginMigrationManager(plugins_dir, 'sqlite://')
manager.verifier = PluginVerifier(plugins_dir, require_signed=True,
trusted_key_paths=[])
assert manager.run_plugin_migrations('demo') is False