Check the installed packages against the lockfiles
A clean lockfile does not mean a clean workstation. pip-audit reported vulnerabilities here while requirements.txt was already pinned to fixed versions - the installed environment had drifted from the pins, and nothing compared the two. The same drift had left requirements-dev.txt uninstalled, so pytest.ini's -n auto failed for want of pytest-xdist. Add scripts/check-dependency-drift.py, comparing installed distributions against both lockfiles. MISSING is reported separately from MISMATCHED: a package that was never installed and a package at the wrong version need different fixes, and folding them together hid the missing dev requirements. Wire it into the style gate advisorially, with DEPENDENCY_DRIFT_ENFORCE=1 for CI. A developer mid-upgrade should not be blocked by a gate that is telling them something they already know. The gate now probes python, python3 and py by executing them rather than by looking them up on PATH. On Windows, python3 resolves to the Microsoft Store alias stub, which is present on PATH and does nothing when run. CONTRIBUTING.md gains the Windows-specific traps this exposed: npm.cmd rather than npm under a Restricted execution policy, the explicit Git Bash path because bare bash finds the WSL shim, and npm test rather than npx vitest run because only the pretest hook stages plugin frontends.
This commit is contained in:
benjamin.baker
137
scripts/check-dependency-drift.py
Normal file
137
scripts/check-dependency-drift.py
Normal file
@@ -0,0 +1,137 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Compare installed package versions against the pinned lockfiles.
|
||||
|
||||
Editing a lockfile changes what a FUTURE install produces. It does not touch an
|
||||
already-populated site-packages. On 2026-08-29 commit a8f8580 raised eight pins
|
||||
carrying advisories; no install was re-run, so this workstation kept running
|
||||
pyjwt 2.12.1 and idna 3.13 for three weeks while requirements.txt said 2.13.0
|
||||
and 3.19. Every local test run in that window exercised different libraries than
|
||||
a container build would have, and a vulnerability scan of the environment
|
||||
disagreed with a scan of the lockfile.
|
||||
|
||||
MISSING is reported separately from MISMATCHED on purpose. The first pass at
|
||||
this incident only compared versions of packages that were installed, which
|
||||
silently skipped pytest-xdist and execnet - absent entirely. That surfaced later
|
||||
as "pytest: error: unrecognized arguments: -n", which reads as a broken
|
||||
pytest.ini rather than a stale environment, and invites deleting "-n auto"
|
||||
instead of installing the plugin.
|
||||
|
||||
Exit codes:
|
||||
0 environment matches the lockfiles
|
||||
1 drift found
|
||||
2 could not run the comparison (bad arguments, unreadable lockfile)
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import importlib.metadata
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
PIN_PATTERN = re.compile(r'^([A-Za-z0-9._-]+)==([^\s\\;]+)')
|
||||
|
||||
DEFAULT_LOCKFILES = ('requirements.txt', 'requirements-dev.txt')
|
||||
|
||||
|
||||
def normalize(name):
|
||||
"""PEP 503 normalization, so Pillow/pillow and pytest_cov/pytest-cov agree."""
|
||||
return re.sub(r'[-_.]+', '-', name).lower()
|
||||
|
||||
|
||||
def read_pins(lockfile):
|
||||
"""Map normalized package name to pinned version for one lockfile.
|
||||
|
||||
uv pip compile emits hashes on continuation lines and comments elsewhere;
|
||||
only lines that START with name==version are pins, which is what the regex
|
||||
anchors on.
|
||||
"""
|
||||
pins = {}
|
||||
for line in lockfile.read_text(encoding='utf-8').splitlines():
|
||||
match = PIN_PATTERN.match(line)
|
||||
if match:
|
||||
pins[normalize(match.group(1))] = match.group(2)
|
||||
return pins
|
||||
|
||||
|
||||
def compare(pins):
|
||||
"""Split pinned packages into missing, mismatched, and satisfied."""
|
||||
missing, mismatched, satisfied = [], [], 0
|
||||
for name, wanted in sorted(pins.items()):
|
||||
try:
|
||||
installed = importlib.metadata.version(name)
|
||||
except importlib.metadata.PackageNotFoundError:
|
||||
missing.append((name, wanted))
|
||||
continue
|
||||
if normalize_version(installed) != normalize_version(wanted):
|
||||
mismatched.append((name, installed, wanted))
|
||||
else:
|
||||
satisfied += 1
|
||||
return missing, mismatched, satisfied
|
||||
|
||||
|
||||
def normalize_version(version):
|
||||
"""Trim a trailing .0 series so 1.2.0 and 1.2 compare equal."""
|
||||
trimmed = version.strip()
|
||||
while trimmed.endswith('.0'):
|
||||
trimmed = trimmed[:-2]
|
||||
return trimmed
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description='Check installed packages against the pinned lockfiles.')
|
||||
parser.add_argument('lockfiles', nargs='*', default=list(DEFAULT_LOCKFILES),
|
||||
help='lockfiles to check (default: %s)'
|
||||
% ' '.join(DEFAULT_LOCKFILES))
|
||||
parser.add_argument('--quiet', action='store_true',
|
||||
help='print only drift, not the clean summary')
|
||||
arguments = parser.parse_args()
|
||||
|
||||
repo_root = Path(__file__).resolve().parent.parent
|
||||
total_missing = total_mismatched = 0
|
||||
report_lines = []
|
||||
|
||||
for name in arguments.lockfiles:
|
||||
lockfile = Path(name)
|
||||
if not lockfile.is_absolute():
|
||||
lockfile = repo_root / lockfile
|
||||
if not lockfile.is_file():
|
||||
print('ERROR: lockfile not found: %s' % lockfile, file=sys.stderr)
|
||||
return 2
|
||||
|
||||
pins = read_pins(lockfile)
|
||||
if not pins:
|
||||
print('ERROR: no pins parsed from %s' % lockfile, file=sys.stderr)
|
||||
return 2
|
||||
|
||||
missing, mismatched, satisfied = compare(pins)
|
||||
total_missing += len(missing)
|
||||
total_mismatched += len(mismatched)
|
||||
|
||||
for package, wanted in missing:
|
||||
report_lines.append(' MISSING %s (pinned %s) [%s]'
|
||||
% (package, wanted, lockfile.name))
|
||||
for package, installed, wanted in mismatched:
|
||||
report_lines.append(' MISMATCH %s installed %s, pinned %s [%s]'
|
||||
% (package, installed, wanted, lockfile.name))
|
||||
|
||||
if not arguments.quiet and not missing and not mismatched:
|
||||
print('OK: %s - %d pins satisfied' % (lockfile.name, satisfied))
|
||||
|
||||
if report_lines:
|
||||
print('FAIL: dependency drift - %d missing, %d mismatched:'
|
||||
% (total_missing, total_mismatched))
|
||||
for line in report_lines:
|
||||
print(line)
|
||||
print('')
|
||||
print('Interpreter: %s' % sys.executable)
|
||||
print('Fix with:')
|
||||
for name in arguments.lockfiles:
|
||||
print(' python -m pip install --require-hashes -r %s' % name)
|
||||
return 1
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -608,6 +608,67 @@ if [ -n "$SITE_HITS" ]; then
|
||||
fi
|
||||
fi
|
||||
|
||||
# --- Dependency drift -------------------------------------------------------
|
||||
#
|
||||
# Not a naming rule, but it belongs to the same question: can this run be
|
||||
# trusted. A stale environment makes every OTHER check here a statement about
|
||||
# libraries that are not the ones the lockfile specifies.
|
||||
#
|
||||
# ADVISORY BY DEFAULT, and deliberately so. Set DEPENDENCY_DRIFT_ENFORCE=1 to
|
||||
# fail the gate (CI should). Two considerations pull against each other here:
|
||||
# a warning nobody reads is worthless, but a blocking check wedges anyone who
|
||||
# is mid-upgrade and legitimately holds an unpinned version for an afternoon.
|
||||
# The tie-breaker is the comment above the site-literal scan: this file has
|
||||
# twice acquired a reasonable-looking flag that silently disabled a check, once
|
||||
# for months with the gate green. A check that ANNOUNCES it is advisory cannot
|
||||
# become that, because its output changes the moment drift appears.
|
||||
#
|
||||
# Skipped when python is absent rather than failing: this script is also run on
|
||||
# machines that only build the frontend.
|
||||
#
|
||||
# Each candidate is TEST-EXECUTED, not merely located with command -v. On
|
||||
# Windows, "python3" resolves to the Microsoft Store app-execution alias - a
|
||||
# zero-byte stub that exists on PATH, satisfies command -v, and then prints
|
||||
# "Python was not found; run without arguments to install from the Microsoft
|
||||
# Store" to stderr and exits non-zero. Locating it and trusting it reported the
|
||||
# drift check as failing on a machine whose environment was in fact clean.
|
||||
DEPENDENCY_DRIFT_ENFORCE=${DEPENDENCY_DRIFT_ENFORCE:-0}
|
||||
echo "==> Checking installed packages against the lockfiles..."
|
||||
DRIFT_PYTHON=""
|
||||
for CANDIDATE in python python3 py; do
|
||||
if command -v "$CANDIDATE" >/dev/null 2>&1 \
|
||||
&& "$CANDIDATE" -c 'import sys' >/dev/null 2>&1; then
|
||||
DRIFT_PYTHON="$CANDIDATE"
|
||||
break
|
||||
fi
|
||||
done
|
||||
# The checker and the lockfiles must both be present. They are not when the gate
|
||||
# is copied alone into a throwaway tree, which is exactly what
|
||||
# tests/test_naming_gate_migrations.py does to exercise the non-git fallback
|
||||
# branch. Absent inputs mean "nothing to compare", not "drift".
|
||||
if [ ! -f scripts/check-dependency-drift.py ] || [ ! -f requirements.txt ]; then
|
||||
DRIFT_PYTHON=""
|
||||
fi
|
||||
if [ -z "$DRIFT_PYTHON" ]; then
|
||||
echo " python not found, skipping dependency drift check"
|
||||
echo
|
||||
else
|
||||
DRIFT_OUTPUT=$("$DRIFT_PYTHON" scripts/check-dependency-drift.py --quiet 2>&1) || DRIFT_STATUS=$?
|
||||
DRIFT_STATUS=${DRIFT_STATUS:-0}
|
||||
if [ "$DRIFT_STATUS" -ne 0 ]; then
|
||||
if [ "$DEPENDENCY_DRIFT_ENFORCE" = "1" ]; then
|
||||
echo "$DRIFT_OUTPUT"
|
||||
echo
|
||||
VIOLATIONS=$((VIOLATIONS + 1))
|
||||
else
|
||||
echo " dependency drift found (report only, not failing):"
|
||||
echo "$DRIFT_OUTPUT" | sed 's/^/ /'
|
||||
echo " Set DEPENDENCY_DRIFT_ENFORCE=1 to make this fail the gate."
|
||||
echo
|
||||
fi
|
||||
fi
|
||||
fi
|
||||
|
||||
if [ "$VIOLATIONS" -gt 0 ]; then
|
||||
echo "=================================================="
|
||||
echo "$VIOLATIONS naming/style violation(s) found."
|
||||
|
||||
Reference in New Issue
Block a user