Offline bundle: pin and hash the webapp wheels, refuse unpinned ones, check for updates

- webapp/requirements.txt pins all eight wheels with sha256, at the fixed
  releases (lxml 6.1.3 for GHSA-vfmq-68hx-4jfw / PYSEC-2026-87, werkzeug
  3.1.9 for GHSA-g6x2-hccm-hh4m).
- download-packages.sh fetches exactly that set for Python 3.12 / x86_64
  into an emptied folder; PIP_DIR is honoured inside the container (own
  mount), files are handed back to the calling user, USE_DOCKER=1 forces
  the container on a noble host, an unknown package name now prints apt's
  error instead of failing silently, and linux-modules-extra-7.0.* is gone
  (it does not exist; 7.0 ships those modules in linux-modules).
- Playbook: the wheel installer refuses any wheel whose sha256 is not
  pinned, needs exactly one wheel per pin, and removes an older installed
  version by its RECORD first. The webapp copy is a full rsync every run
  (was cp with creates: app.py, which never updated an install).
- scripts/check-updates.py: offline debs against noble security/updates
  and the pins against PyPI + osv.dev; exit 1 when something needs
  updating. Run it before building a USB.

Tested: installer in a Python 3.12 venv (clean install, upgrade over lxml
6.0.4 + werkzeug 3.1.8 leaves no old dist-info, re-run is a no-op, a stray
wheel is refused); check-updates.py flags the April bundle (94 security
updates) and passes the pins.
This commit is contained in:
cproudlock
2026-10-15 09:41:16 -04:00
parent ead3314f58
commit 18f20b00e2
4 changed files with 225 additions and 21 deletions

View File

@@ -1057,7 +1057,7 @@
files (vmlinuz-ubuntu-{{ blancco_ksuffix }},
kexec-initrd-{{ blancco_ksuffix }}.img,
kmod-{{ blancco_kver }}.tar.gz) are NOT built. Install
linux-image-{{ blancco_kver }} + linux-modules-extra-{{ blancco_kver }}
linux-image-{{ blancco_kver }} + linux-modules-{{ blancco_kver }}
(scripts/download-packages.sh, BLANCCO_KVER) or set blancco_kver
to an installed version (ls /lib/modules), then re-run.
when: not blancco_kernel_ok
@@ -1445,10 +1445,15 @@
mode: '0755'
- name: "Copy webapp from USB"
shell: >
cp -r "{{ usb_root }}/webapp/"* /opt/pxe-webapp/ 2>/dev/null || true
# The whole tree (app, config, services, views, templates, static), every
# run, so a re-run with a newer USB updates the app. Was: cp with
# creates: app.py, which never updated an existing install.
shell: |
set -e
test -f "{{ usb_root }}/webapp/app.py"
rsync -a --delete --exclude __pycache__ "{{ usb_root }}/webapp/" /opt/pxe-webapp/
args:
creates: /opt/pxe-webapp/app.py
executable: /bin/bash
- name: "Install webapp Python dependencies (offline wheels)"
args:
@@ -1465,16 +1470,51 @@
if [ -n "$WHEEL_DIR" ]; then
# Install wheels directly using Python zipfile (bypasses pip entirely)
# This avoids the pip3/distutils incompatibility between Ubuntu 22.04 debs and Python 3.12
export REQS="{{ usb_root }}/webapp/requirements.txt"
python3 << 'PYEOF'
import zipfile, sysconfig, glob, os
# Installs only the pinned wheels: the sha256 of every wheel must be in
# webapp/requirements.txt, and each pinned package must have exactly one
# wheel. An older installed version of a package is removed first (by its
# RECORD), so an upgrade does not leave the old files beside the new.
import zipfile, sysconfig, glob, os, re, sys, hashlib, csv, shutil
site = sysconfig.get_path('platlib')
wheel_dir = os.environ['WHEEL_DIR']
for whl in sorted(glob.glob(os.path.join(wheel_dir, '*.whl'))):
name = os.path.basename(whl)
print(f' Installing {name}')
reqs = open(os.environ['REQS']).read()
pins, hashes = {}, set()
for m in re.finditer(r'^([A-Za-z0-9_.-]+)==([^\s\\]+)((?:\s*\\?\s*--hash=sha256:[0-9a-f]{64})+)', reqs, re.M):
name = re.sub(r'[-_.]+', '_', m.group(1)).lower()
pins[name] = m.group(2)
hashes.update(re.findall(r'sha256:([0-9a-f]{64})', m.group(3)))
wheels = sorted(glob.glob(os.path.join(wheel_dir, '*.whl')))
seen = {}
for whl in wheels:
h = hashlib.sha256(open(whl, 'rb').read()).hexdigest()
name = re.sub(r'[-_.]+', '_', os.path.basename(whl).split('-')[0]).lower()
if h not in hashes or name not in pins:
sys.exit(f'REFUSED: {os.path.basename(whl)} is not a pinned wheel (sha256 {h[:16]}...)')
seen.setdefault(name, []).append(whl)
bad = [n for n in pins if len(seen.get(n, [])) != 1]
if bad:
sys.exit(f'REFUSED: need exactly one wheel for each pinned package; check: {", ".join(bad)}')
for name, (whl,) in sorted(seen.items()):
for old in glob.glob(os.path.join(site, '*.dist-info')):
stem = os.path.basename(old)[:-len('.dist-info')]
oname = re.sub(r'[-_.]+', '_', stem.rsplit('-', 1)[0]).lower()
if oname != name:
continue
rec = os.path.join(old, 'RECORD')
if os.path.isfile(rec):
for row in csv.reader(open(rec)):
if not row:
continue
f = os.path.normpath(os.path.join(site, row[0]))
if f.startswith(site + os.sep) and os.path.isfile(f):
os.remove(f)
shutil.rmtree(old, ignore_errors=True)
print(f' Installing {os.path.basename(whl)}')
with zipfile.ZipFile(whl) as z:
z.extractall(site)
print('All wheels installed to ' + site)
print('All pinned wheels installed to ' + site)
PYEOF
else
# Fallback: try system pip (works if system has internet and compatible pip)