Capture site content from a live server so a new one can build every image

A freshly installed server could not build standard or engineer: about
8 GB they install from existed only on the running server.
scripts/capture-site-content.sh pulls it over SSH into the gitignored
payload:
- _shared/Applications/extra (Office, Adobe, OpenText, Java, fonts, Wi-Fi,
  Zscaler), without pxe-status/ (the playbook deploys that);
- each image's own Deploy/Applications files and folders (GE CA
  certificates, the Engineering Layer payload), without the repo-managed
  Install-EngineeringLayer.ps1 and without stale BPRT copies;
- the links images keep there, as links.txt;
- fresh bios-staging (*.exe plus models.txt, which the Dell update
  rewrites), boot.wim, and the known Blancco file set.

It dry-runs by default, compares by checksum (a Blancco config.img had the
same size and time as the stale copy), writes SHA256SUMS, and refuses what
FAT32 cannot hold: files of 4 GiB or more, Windows-illegal names, and
case-only name clashes.

build-usb.sh copies site-content/ unsplit and refuses to build without it
(unless --skip-site-content), or when SHA256SUMS is missing or does not
cover every file. The playbook checks SHA256SUMS, then merges the content
into winpeapps and recreates the links. It only links to folders that
exist, so it never creates an empty _shared BPRT, which would stop the
first MCL import from seeding it. Link lines are validated before anything
is copied.

pull-bios.sh is gone. It read a path the server no longer uses, with a
hard-coded password.
This commit is contained in:
cproudlock
2026-10-16 15:17:42 -04:00
parent d63cc5761b
commit 9af1c6d89a
8 changed files with 381 additions and 27 deletions

View File

@@ -24,12 +24,16 @@
# --image FILE SIZE build a disk image file instead of writing a device
# --payload-root DIR take the gitignored payload dirs (offline-packages/,
# pip-wheels/, boot-files/, boot-tools/, enrollment/,
# bios-staging/, drivers-staging/) from DIR instead of
# bios-staging/, drivers-staging/, site-content/) from DIR instead of
# the repo root (e.g. a main checkout next to a worktree)
# --skip-enrollment do not copy enrollment/
# --skip-bios do not copy bios-staging/
# --skip-drivers do not copy drivers-staging/
# --skip-boot-tools do not copy boot-tools/
# --skip-site-content do not copy site-content/ (Office, Adobe, certs,
# Engineering Layer...: scripts/capture-site-content.sh).
# Without it a new server cannot build standard or
# engineer until that content is copied over by hand.
# --site-vars FILE put FILE on CIDATA as playbook/site-vars.yml (the
# per-site NIC MACs; see playbook/site-vars.example.yml)
# --no-site-vars build without site vars (the first-boot playbook then
@@ -58,7 +62,7 @@ usage() {
echo " $0 [options] --image out.img SIZE|auto /path/to/ubuntu-24.04.iso [/path/to/winpe-images]"
echo ""
echo " Options: --payload-root DIR, --site-vars FILE | --no-site-vars, --skip-enrollment,"
echo " --skip-bios, --skip-drivers, --skip-boot-tools, --yes"
echo " --skip-bios, --skip-drivers, --skip-boot-tools, --skip-site-content, --yes"
echo ""
echo " The optional last argument is the path to WinPE deployment content"
echo " (e.g., the mounted Media Creator LITE USB). If provided, the images"
@@ -77,6 +81,7 @@ SKIP_ENROLLMENT=0
SKIP_BIOS=0
SKIP_DRIVERS=0
SKIP_BOOT_TOOLS=0
SKIP_SITE_CONTENT=0
ASSUME_YES=0
SITE_VARS=""
NO_SITE_VARS=0
@@ -97,6 +102,7 @@ while [ $# -gt 0 ]; do
--skip-bios) SKIP_BIOS=1; shift ;;
--skip-drivers) SKIP_DRIVERS=1; shift ;;
--skip-boot-tools) SKIP_BOOT_TOOLS=1; shift ;;
--skip-site-content) SKIP_SITE_CONTENT=1; shift ;;
--yes|-y) ASSUME_YES=1; shift ;;
-h|--help) usage; exit 0 ;;
--) shift; POSITIONAL+=("$@"); break ;;
@@ -124,10 +130,12 @@ BOOT_TOOLS_DIR="$PAYLOAD_ROOT/boot-tools"
ENROLLMENT_DIR="$PAYLOAD_ROOT/enrollment"
BIOS_DIR="$PAYLOAD_ROOT/bios-staging"
DRIVERS_DIR="$PAYLOAD_ROOT/drivers-staging"
SITE_CONTENT_DIR="$PAYLOAD_ROOT/site-content"
[ "$SKIP_ENROLLMENT" -eq 1 ] && ENROLLMENT_DIR=""
[ "$SKIP_BIOS" -eq 1 ] && BIOS_DIR=""
[ "$SKIP_DRIVERS" -eq 1 ] && DRIVERS_DIR=""
[ "$SKIP_BOOT_TOOLS" -eq 1 ] && BOOT_TOOLS_DIR=""
[ "$SKIP_SITE_CONTENT" -eq 1 ] && SITE_CONTENT_DIR=""
# FAT32 has a 4GB max file size; files larger than this are split into
# chunks that the playbook reassembles with `cat`.
@@ -299,6 +307,34 @@ else
echo " cannot check it was resolved against $ISO_VOLID_NOW."
fi
# site-content/ is restored file by file against its SHA256SUMS, so it is
# never split: a file FAT32 cannot hold refuses the build instead.
if [ -n "$SITE_CONTENT_DIR" ]; then
if [ ! -d "$SITE_CONTENT_DIR" ]; then
echo "ERROR: no site-content/ in $PAYLOAD_ROOT. Run scripts/capture-site-content.sh --apply"
echo " (or pass --skip-site-content: the new server then lacks Office, Adobe, certs"
echo " and the Engineering Layer until they are copied by hand)."
exit 1
fi
if [ ! -f "$SITE_CONTENT_DIR/SHA256SUMS" ]; then
echo "ERROR: site-content/SHA256SUMS missing. Re-run scripts/capture-site-content.sh --apply"
exit 1
fi
SC_BIG=$(find -L "$SITE_CONTENT_DIR" -type f -size +4194303k)
if [ -n "$SC_BIG" ]; then
echo "ERROR: site-content/ has files of 4 GiB or more (FAT32 cannot hold them):"
printf '%s\n' "$SC_BIG" | sed 's/^/ /'
exit 1
fi
SC_LISTED=$(grep -c . "$SITE_CONTENT_DIR/SHA256SUMS")
SC_FOUND=$(find -L "$SITE_CONTENT_DIR" -type f ! -name SHA256SUMS | wc -l)
if [ "$SC_LISTED" -ne "$SC_FOUND" ]; then
echo "ERROR: site-content/ holds $SC_FOUND files but SHA256SUMS lists $SC_LISTED."
echo " Re-run scripts/capture-site-content.sh --apply"
exit 1
fi
fi
# Verify required source files exist
for f in "$AUTOINSTALL_DIR/user-data" "$AUTOINSTALL_DIR/meta-data" "$PLAYBOOK_DIR/pxe_server_setup.yml"; do
if [ ! -f "$f" ]; then
@@ -455,7 +491,7 @@ if [ "$IMAGE_MODE" -eq 1 ]; then
PAYLOAD_BYTES=$(src_bytes "$AUTOINSTALL_DIR/user-data" "$AUTOINSTALL_DIR/meta-data" \
"$OFFLINE_PKG_DIR" "$PLAYBOOK_DIR" "$WEBAPP_DIR" "$PIP_WHEELS_DIR" \
"$BOOT_FILES_DIR" "$BOOT_TOOLS_DIR" "$ENROLLMENT_DIR" "$BIOS_DIR" \
"$DRIVERS_DIR" "$WINPE_SOURCE")
"$DRIVERS_DIR" "$SITE_CONTENT_DIR" "$WINPE_SOURCE")
if [ "$IMAGE_SIZE_ARG" = "auto" ]; then
# payload + 10% + 256 MiB FAT/cluster slack, rounded up to 1 MiB
CIDATA_BYTES=$(( PAYLOAD_BYTES + PAYLOAD_BYTES / 10 + 256 * 1024 * 1024 ))
@@ -710,6 +746,15 @@ else
echo " No enrollment/ directory found (PPKGs can be uploaded via webapp later)"
fi
# Copy the site content (capture-site-content.sh); the playbook checks
# SHA256SUMS before it restores anything.
if [ -n "$SITE_CONTENT_DIR" ]; then
t_cp "" "$SITE_CONTENT_DIR"
echo " Copied site-content/ ($(du -shL "$SITE_CONTENT_DIR" | cut -f1), $SC_FOUND files)"
else
echo " Skipped site-content/ (--skip-site-content)"
fi
# Copy BIOS update binaries if staged
if [ -n "$BIOS_DIR" ] && [ -d "$BIOS_DIR" ] && [ "$(ls -A "$BIOS_DIR" 2>/dev/null)" ]; then
echo " Copying BIOS update binaries from bios-staging/..."

237
scripts/capture-site-content.sh Executable file
View File

@@ -0,0 +1,237 @@
#!/bin/bash
#
# capture-site-content.sh - Pull the content a PXE server needs but neither
# the repo nor the MCL image import provides, into the gitignored USB payload,
# so a freshly installed server can build every image from the USB alone.
#
# Run on the build PC (not the server), with the imaging LAN reachable:
# scripts/capture-site-content.sh show what would change
# scripts/capture-site-content.sh --apply pull it
#
# What it pulls (live path -> payload dir):
# winpeapps/_shared/Applications/extra/ -> site-content/shared/Applications/extra/
# Office, Adobe, OpenText, Java, fonts, Wi-Fi profile, print drivers,
# Zscaler: every unattend installs from C:\Deploy\Applications\extra.
# pxe-status/ is left out (the playbook deploys Send-ImagingStage.ps1).
# winpeapps/<image>/Deploy/Applications/ top-level files (GE CA and Zscaler
# certificates) and real folders (extra_engineering)
# -> site-content/images/<image>/Applications/
# Install-EngineeringLayer.ps1 is left out (repo playbook/engineering/).
# the links each image keeps in Deploy/Applications (extra, BPRT)
# -> site-content/links.txt
# winpeapps/_shared/BIOS/*.exe + models.txt -> bios-staging/
# models.txt pairs each model with its file; the webapp's Dell update
# rewrites it, so the live copy, not the repo's, matches the binaries.
# www/win11/sources/boot.wim -> boot-files/boot.wim
# www/blancco/<the files already in boot-tools/blancco/>
# -> boot-tools/blancco/
# only that known set: kexec-initrd, kmod, preferences.xml and the rest
# are built by the playbook on the server.
#
# Not pulled, on purpose: BPRT (the first MCL import seeds it, plus our
# overlay from the repo), Operating Systems / Packages / drivers (the MCL
# import and Upload-Image.ps1 -IncludeDrivers bring those), Blancco reports,
# Clonezilla backups and imaging logs (records, not setup).
#
# Then it writes site-content/SHA256SUMS (the playbook checks it before it
# restores anything), and refuses content a FAT32 USB cannot hold: a file of
# 4 GiB or more, a name Windows rejects, or two names that differ only in case.
#
# Needs SSH key login as pxe on the server (files there are world-readable).
# Folders are mirrored (--delete): what the server no longer has goes from
# the payload too. Nothing on the server is changed.
set -euo pipefail
REPO_ROOT="$(cd "$(dirname "$0")"/.. && pwd)"
SERVER="172.16.9.1"
USER_="pxe"
PAYLOAD_ROOT="$REPO_ROOT"
APPLY=0
usage() {
echo "Usage: $0 [--apply] [--server IP] [--user NAME] [--payload-root DIR]"
echo " Without --apply it only shows what would change."
}
while [ $# -gt 0 ]; do
case "$1" in
--apply) APPLY=1; shift ;;
--server) SERVER="${2:?}"; shift 2 ;;
--user) USER_="${2:?}"; shift 2 ;;
--payload-root) PAYLOAD_ROOT="$(cd "${2:?}" && pwd)"; shift 2 ;;
-h|--help) usage; exit 0 ;;
*) usage; exit 1 ;;
esac
done
SHARE="${CAPTURE_SHARE:-/srv/samba/winpeapps}" # overridable for tests only
WWW="${CAPTURE_WWW:-/var/www/html}"
SITE="$PAYLOAD_ROOT/site-content"
BIOS="$PAYLOAD_ROOT/bios-staging"
BOOT_FILES="$PAYLOAD_ROOT/boot-files"
BLANCCO="$PAYLOAD_ROOT/boot-tools/blancco"
SSH_OPTS=(-o BatchMode=yes -o ConnectTimeout=10)
REMOTE="$USER_@$SERVER"
DRY=(-n)
[ "$APPLY" -eq 1 ] && DRY=()
# rsync -a over ssh. --checksum: the same size and time is not proof of the
# same content (a Blancco config.img differed that way). -L: a symlink in a
# pulled folder becomes its file (FAT32 has no links). Itemized, so the dry
# run lists every change.
pull() {
rsync -aL --checksum --mkpath --delete --itemize-changes --human-readable --stats \
-e "ssh ${SSH_OPTS[*]}" "${DRY[@]}" "$@" | summarize
}
# Itemized lines are noisy for 7 GB of Office; print counts and the first few.
summarize() {
awk '
/^[<>ch.*]/ || /^\*deleting/ { n++; if (n <= 8) print " " $0; next }
/^Number of (regular )?files transferred|^Total transferred file size|^Total file size/ { s = s " " $0 "\n" }
END { if (n > 8) printf " ... %d changes in all\n", n; else if (n == 0) print " (no changes)"; printf "%s", s }'
}
echo "Server : $REMOTE"
echo "Payload: $PAYLOAD_ROOT"
[ "$APPLY" -eq 1 ] && echo "Mode : APPLY" || echo "Mode : dry run (add --apply to pull)"
echo
ssh "${SSH_OPTS[@]}" "$REMOTE" true || { echo "ERROR: cannot ssh to $REMOTE (key login needed)" >&2; exit 1; }
# --- what the server has ------------------------------------------------------
# One remote listing: images with Deploy/Applications, and per image its
# top-level entries as "image<TAB>kind<TAB>name<TAB>link target".
LISTING=$(ssh "${SSH_OPTS[@]}" "$REMOTE" "
cd '$SHARE' || exit 1
for apps in */Deploy/Applications; do
img=\${apps%%/*}
case \"\$img\" in _*) continue ;; esac
[ -d \"\$apps\" ] || continue
for e in \"\$apps\"/* \"\$apps\"/.[!.]*; do
[ -e \"\$e\" ] || [ -L \"\$e\" ] || continue
n=\${e##*/}
if [ -L \"\$e\" ]; then printf '%s\tlink\t%s\t%s\n' \"\$img\" \"\$n\" \"\$(readlink \"\$e\")\"
elif [ -d \"\$e\" ]; then printf '%s\tdir\t%s\t\n' \"\$img\" \"\$n\"
else printf '%s\tfile\t%s\t\n' \"\$img\" \"\$n\"; fi
done
done")
IMAGES=$(printf '%s\n' "$LISTING" | cut -f1 | sort -u | grep -v '^$' || true)
[ -n "$IMAGES" ] || { echo "ERROR: no image on $SERVER has Deploy/Applications" >&2; exit 1; }
# Links into _shared are recreated by the playbook. Anything else is reported.
LINKS=""
while IFS=$'\t' read -r img kind name target; do
[ "$kind" = link ] || continue
case "$target" in
"$SHARE"/_shared/*) LINKS+="$img $name ${target#"$SHARE"/}"$'\n' ;;
*) echo "WARNING: $img/Deploy/Applications/$name links outside _shared ($target); not captured" ;;
esac
done <<< "$LISTING"
# --- 1. _shared/Applications/extra -------------------------------------------
echo "[1/5] _shared/Applications/extra -> site-content/shared/Applications/extra/"
pull --exclude=/pxe-status/ "$REMOTE:$SHARE/_shared/Applications/extra/" "$SITE/shared/Applications/extra/"
# --- 2. per-image Applications content ---------------------------------------
echo "[2/5] per-image Deploy/Applications files and folders -> site-content/images/"
for img in $IMAGES; do
names=$(printf '%s\n' "$LISTING" | awk -F'\t' -v i="$img" '$1 == i && ($2 == "file" || $2 == "dir") { print $3 }')
# BPRT as a real folder is an old import's copy of the vendor folder: not ours.
names=$(printf '%s\n' "$names" | grep -vix 'bprt' || true)
if [ -z "$names" ]; then
echo " $img: nothing (links only)"
[ "$APPLY" -eq 1 ] && rm -rf "$SITE/images/$img"
continue
fi
echo " $img: $(printf '%s\n' "$names" | tr '\n' ' ')"
filter=()
while IFS= read -r n; do
[ -n "$n" ] && filter+=(--include="/$n" --include="/$n/***")
done <<< "$names"
pull --exclude=Install-EngineeringLayer.ps1 "${filter[@]}" --exclude='*' \
--delete-excluded "$REMOTE:$SHARE/$img/Deploy/Applications/" "$SITE/images/$img/Applications/"
done
# --- 3. BIOS --------------------------------------------------------------------
echo "[3/5] _shared/BIOS (*.exe, models.txt) -> bios-staging/"
pull --include='/*.exe' --include=/models.txt --exclude='*' --delete-excluded \
"$REMOTE:$SHARE/_shared/BIOS/" "$BIOS/"
# --- 4. boot.wim -----------------------------------------------------------------
echo "[4/5] win11/sources/boot.wim -> boot-files/boot.wim"
pull "$REMOTE:$WWW/win11/sources/boot.wim" "$BOOT_FILES/boot.wim"
# --- 5. Blancco (the known set only) ---------------------------------------------
echo "[5/5] blancco (files already in boot-tools/blancco/) -> boot-tools/blancco/"
if [ -d "$BLANCCO" ] && [ -n "$(ls -A "$BLANCCO")" ]; then
filter=()
for e in "$BLANCCO"/*; do
n=${e##*/}
filter+=(--include="/$n")
[ -d "$e" ] && filter+=(--include="/$n/***")
done
pull "${filter[@]}" --exclude='*' "$REMOTE:$WWW/blancco/" "$BLANCCO/"
else
echo " SKIP: boot-tools/blancco/ is empty (run scripts/prepare-boot-tools.sh first)"
fi
if [ "$APPLY" -ne 1 ]; then
echo
echo "Dry run: nothing written. Re-run with --apply to pull."
exit 0
fi
# --- links, checks, checksums ---------------------------------------------------
printf '# image<TAB>name in Deploy/Applications<TAB>link target under winpeapps/\n%s' "$LINKS" > "$SITE/links.txt"
echo
echo "Checking what a FAT32 USB can hold..."
BAD=0
BIG=$(find "$SITE" "$BIOS" "$BOOT_FILES/boot.wim" -type f -size +4194303k 2>/dev/null || true)
if [ -n "$BIG" ]; then
echo "ERROR: files of 4 GiB or more (FAT32 cannot hold them):"; printf '%s\n' "$BIG" | sed 's/^/ /'; BAD=1
fi
ILLEGAL=$(cd "$SITE" && find . -name '*[<>:"|?*\\]*' 2>/dev/null || true)
if [ -n "$ILLEGAL" ]; then
echo "ERROR: names Windows/FAT32 rejects:"; printf '%s\n' "$ILLEGAL" | sed 's/^/ /'; BAD=1
fi
CASE=$(cd "$SITE" && find . | sort -f | uniq -di || true)
if [ -n "$CASE" ]; then
echo "ERROR: names that differ only in case (one would overwrite the other):"
printf '%s\n' "$CASE" | sed 's/^/ /'; BAD=1
fi
[ "$BAD" -eq 0 ] || exit 1
echo " ok"
echo "Writing site-content/SHA256SUMS..."
# Written beside site-content, not in it, so the list never names itself.
( cd "$SITE" && find . -type f ! -name SHA256SUMS -print0 | sort -z | xargs -0 sha256sum ) > "$PAYLOAD_ROOT/.site-content.SHA256SUMS.tmp"
mv "$PAYLOAD_ROOT/.site-content.SHA256SUMS.tmp" "$SITE/SHA256SUMS"
echo
echo "Summary:"
printf ' site-content/ %s, %s files\n' "$(du -sh "$SITE" | cut -f1)" "$(grep -c . "$SITE/SHA256SUMS")"
printf ' links.txt %s links\n' "$(grep -vc '^#' "$SITE/links.txt" || true)"
printf ' bios-staging/ %s, %s .exe\n' "$(du -sh "$BIOS" | cut -f1)" "$(find "$BIOS" -name '*.exe' | wc -l)"
printf ' boot.wim %s\n' "$(du -h "$BOOT_FILES/boot.wim" | cut -f1)"
# Files the repo tracks that the server has a different copy of: review and
# commit (or revert) before building the USB.
cd "$REPO_ROOT"
if [ "$PAYLOAD_ROOT" = "$REPO_ROOT" ] && ! git diff --quiet -- boot-tools/blancco/grub-blancco.cfg 2>/dev/null; then
echo
echo "NOTE: boot-tools/blancco/grub-blancco.cfg (tracked) now holds the server's copy:"
git --no-pager diff --stat -- boot-tools/blancco/grub-blancco.cfg
fi
for f in check-bios.cmd hardware-report.ps1; do
if ssh "${SSH_OPTS[@]}" "$REMOTE" "cat '$SHARE/_shared/BIOS/$f'" 2>/dev/null \
| cmp -s - "playbook/shopfloor-setup/BIOS/$f"; then :; else
echo "NOTE: the server's _shared/BIOS/$f differs from playbook/shopfloor-setup/BIOS/$f (the repo copy is installed)"
fi
done
echo
echo "Done. Build the USB next (scripts/build-usb.sh copies site-content/)."

View File

@@ -21,7 +21,7 @@ SCP="sshpass -p $PXE_PASS scp -o StrictHostKeyChecking=no -o ConnectTimeout=10"
# Verify sources exist
if [ ! -d "$BIOS_DIR" ] || [ -z "$(ls -A "$BIOS_DIR" 2>/dev/null)" ]; then
echo "ERROR: bios-staging/ is empty or missing. Run ./pull-bios.sh first."
echo "ERROR: bios-staging/ is empty or missing. Run scripts/capture-site-content.sh --apply first."
exit 1
fi

View File

@@ -1,21 +0,0 @@
#!/bin/bash
# pull-bios.sh - Pull BIOS update binaries from prod PXE server to bios-staging/
# Run this with the USB NIC plugged in, before building the USB.
set -e
REPO_ROOT="$(cd "$(dirname "$0")"/.. && pwd)"
DEST="$REPO_ROOT/bios-staging"
PXE_SERVER="172.16.9.1"
PXE_USER="pxe"
PXE_PASS="pxe"
mkdir -p "$DEST"
echo "Pulling BIOS binaries from $PXE_SERVER..."
sshpass -p "$PXE_PASS" scp -o StrictHostKeyChecking=no -o ConnectTimeout=10 \
"$PXE_USER@$PXE_SERVER:/srv/samba/enrollment/BIOS/*.exe" "$DEST/"
COUNT=$(find "$DEST" -name '*.exe' | wc -l)
SIZE=$(du -sh "$DEST" | cut -f1)
echo "Done: $COUNT files ($SIZE) in bios-staging/"