ge-aerospace/pxe-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Capture site content from a live server so a new one can build every image

Browse Source 
A freshly installed server could not build standard or engineer: about
8 GB they install from existed only on the running server.
scripts/capture-site-content.sh pulls it over SSH into the gitignored
payload:
- _shared/Applications/extra (Office, Adobe, OpenText, Java, fonts, Wi-Fi,
  Zscaler), without pxe-status/ (the playbook deploys that);
- each image's own Deploy/Applications files and folders (GE CA
  certificates, the Engineering Layer payload), without the repo-managed
  Install-EngineeringLayer.ps1 and without stale BPRT copies;
- the links images keep there, as links.txt;
- fresh bios-staging (*.exe plus models.txt, which the Dell update
  rewrites), boot.wim, and the known Blancco file set.

It dry-runs by default, compares by checksum (a Blancco config.img had the
same size and time as the stale copy), writes SHA256SUMS, and refuses what
FAT32 cannot hold: files of 4 GiB or more, Windows-illegal names, and
case-only name clashes.

build-usb.sh copies site-content/ unsplit and refuses to build without it
(unless --skip-site-content), or when SHA256SUMS is missing or does not
cover every file. The playbook checks SHA256SUMS, then merges the content
into winpeapps and recreates the links. It only links to folders that
exist, so it never creates an empty _shared BPRT, which would stop the
first MCL import from seeding it. Link lines are validated before anything
is copied.

pull-bios.sh is gone. It read a path the server no longer uses, with a
hard-coded password.
This commit is contained in:
cproudlock
2026-10-16 15:17:42 -04:00
parent d63cc5761b
commit 9af1c6d89a
8 changed files with 381 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files

51
scripts/build-usb.sh
View File

@@ -24,12 +24,16 @@
# --image FILE SIZE build a disk image file instead of writing a device
# --payload-root DIR take the gitignored payload dirs (offline-packages/,
# pip-wheels/, boot-files/, boot-tools/, enrollment/,
# bios-staging/, drivers-staging/) from DIR instead of
# bios-staging/, drivers-staging/, site-content/) from DIR instead of
# the repo root (e.g. a main checkout next to a worktree)
# --skip-enrollment do not copy enrollment/
# --skip-bios do not copy bios-staging/
# --skip-drivers do not copy drivers-staging/
# --skip-boot-tools do not copy boot-tools/
# --skip-site-content do not copy site-content/ (Office, Adobe, certs,
# Engineering Layer...: scripts/capture-site-content.sh).
# Without it a new server cannot build standard or
# engineer until that content is copied over by hand.
# --site-vars FILE put FILE on CIDATA as playbook/site-vars.yml (the
# per-site NIC MACs; see playbook/site-vars.example.yml)
# --no-site-vars build without site vars (the first-boot playbook then
@@ -58,7 +62,7 @@ usage() {
echo " $0 [options] --image out.img SIZE|auto /path/to/ubuntu-24.04.iso [/path/to/winpe-images]"
echo ""
echo " Options: --payload-root DIR, --site-vars FILE | --no-site-vars, --skip-enrollment,"
echo " --skip-bios, --skip-drivers, --skip-boot-tools, --yes"
echo " --skip-bios, --skip-drivers, --skip-boot-tools, --skip-site-content, --yes"
echo ""
echo " The optional last argument is the path to WinPE deployment content"
echo " (e.g., the mounted Media Creator LITE USB). If provided, the images"
@@ -77,6 +81,7 @@ SKIP_ENROLLMENT=0
SKIP_BIOS=0
SKIP_DRIVERS=0
SKIP_BOOT_TOOLS=0
SKIP_SITE_CONTENT=0
ASSUME_YES=0
SITE_VARS=""
NO_SITE_VARS=0
@@ -97,6 +102,7 @@ while [ $# -gt 0 ]; do
--skip-bios) SKIP_BIOS=1; shift ;;
--skip-drivers) SKIP_DRIVERS=1; shift ;;
--skip-boot-tools) SKIP_BOOT_TOOLS=1; shift ;;
--skip-site-content) SKIP_SITE_CONTENT=1; shift ;;
--yes|-y) ASSUME_YES=1; shift ;;
-h|--help) usage; exit 0 ;;
--) shift; POSITIONAL+=("$@"); break ;;
@@ -124,10 +130,12 @@ BOOT_TOOLS_DIR="$PAYLOAD_ROOT/boot-tools"
ENROLLMENT_DIR="$PAYLOAD_ROOT/enrollment"
BIOS_DIR="$PAYLOAD_ROOT/bios-staging"
DRIVERS_DIR="$PAYLOAD_ROOT/drivers-staging"
SITE_CONTENT_DIR="$PAYLOAD_ROOT/site-content"
[ "$SKIP_ENROLLMENT" -eq 1 ] && ENROLLMENT_DIR=""
[ "$SKIP_BIOS" -eq 1 ] && BIOS_DIR=""
[ "$SKIP_DRIVERS" -eq 1 ] && DRIVERS_DIR=""
[ "$SKIP_BOOT_TOOLS" -eq 1 ] && BOOT_TOOLS_DIR=""
[ "$SKIP_SITE_CONTENT" -eq 1 ] && SITE_CONTENT_DIR=""
# FAT32 has a 4GB max file size; files larger than this are split into
# chunks that the playbook reassembles with `cat`.
@@ -299,6 +307,34 @@ else
echo " cannot check it was resolved against $ISO_VOLID_NOW."
fi
# site-content/ is restored file by file against its SHA256SUMS, so it is
# never split: a file FAT32 cannot hold refuses the build instead.
if [ -n "$SITE_CONTENT_DIR" ]; then
if [ ! -d "$SITE_CONTENT_DIR" ]; then
echo "ERROR: no site-content/ in $PAYLOAD_ROOT. Run scripts/capture-site-content.sh --apply"
echo " (or pass --skip-site-content: the new server then lacks Office, Adobe, certs"
echo " and the Engineering Layer until they are copied by hand)."
exit 1
fi
if [ ! -f "$SITE_CONTENT_DIR/SHA256SUMS" ]; then
echo "ERROR: site-content/SHA256SUMS missing. Re-run scripts/capture-site-content.sh --apply"
exit 1
fi
SC_BIG=$(find -L "$SITE_CONTENT_DIR" -type f -size +4194303k)
if [ -n "$SC_BIG" ]; then
echo "ERROR: site-content/ has files of 4 GiB or more (FAT32 cannot hold them):"
printf '%s\n' "$SC_BIG" | sed 's/^/ /'
exit 1
fi
SC_LISTED=$(grep -c . "$SITE_CONTENT_DIR/SHA256SUMS")
SC_FOUND=$(find -L "$SITE_CONTENT_DIR" -type f ! -name SHA256SUMS | wc -l)
if [ "$SC_LISTED" -ne "$SC_FOUND" ]; then
echo "ERROR: site-content/ holds $SC_FOUND files but SHA256SUMS lists $SC_LISTED."
echo " Re-run scripts/capture-site-content.sh --apply"
exit 1
fi
fi
# Verify required source files exist
for f in "$AUTOINSTALL_DIR/user-data" "$AUTOINSTALL_DIR/meta-data" "$PLAYBOOK_DIR/pxe_server_setup.yml"; do
if [ ! -f "$f" ]; then
@@ -455,7 +491,7 @@ if [ "$IMAGE_MODE" -eq 1 ]; then
PAYLOAD_BYTES=$(src_bytes "$AUTOINSTALL_DIR/user-data" "$AUTOINSTALL_DIR/meta-data" \
"$OFFLINE_PKG_DIR" "$PLAYBOOK_DIR" "$WEBAPP_DIR" "$PIP_WHEELS_DIR" \
"$BOOT_FILES_DIR" "$BOOT_TOOLS_DIR" "$ENROLLMENT_DIR" "$BIOS_DIR" \
"$DRIVERS_DIR" "$WINPE_SOURCE")
"$DRIVERS_DIR" "$SITE_CONTENT_DIR" "$WINPE_SOURCE")
if [ "$IMAGE_SIZE_ARG" = "auto" ]; then
# payload + 10% + 256 MiB FAT/cluster slack, rounded up to 1 MiB
CIDATA_BYTES=$(( PAYLOAD_BYTES + PAYLOAD_BYTES / 10 + 256 * 1024 * 1024 ))
@@ -710,6 +746,15 @@ else
echo " No enrollment/ directory found (PPKGs can be uploaded via webapp later)"
fi
# Copy the site content (capture-site-content.sh); the playbook checks
# SHA256SUMS before it restores anything.
if [ -n "$SITE_CONTENT_DIR" ]; then
t_cp "" "$SITE_CONTENT_DIR"
echo " Copied site-content/ ($(du -shL "$SITE_CONTENT_DIR" | cut -f1), $SC_FOUND files)"
else
echo " Skipped site-content/ (--skip-site-content)"
fi
# Copy BIOS update binaries if staged
if [ -n "$BIOS_DIR" ] && [ -d "$BIOS_DIR" ] && [ "$(ls -A "$BIOS_DIR" 2>/dev/null)" ]; then
echo " Copying BIOS update binaries from bios-staging/..."