Offline install that lands fully patched: resolve against the ISO's own base

- autoinstall: dead mirror + fallback offline-install, so the installed base is exactly the ISO's; late-commands run offline-apt-install.sh on CIDATA packages/ (local apt repo, one transaction, errors fail the install, log /var/log/offline-apt-install.log); first boot no longer runs dpkg -i; real archive sources written back at the end. - download-packages.sh --iso: unpacks the ISO's default install source into a docker image (pxe-iso-base:<version>) and downloads there: all updates of the base, the explicit list (zip included) and the newest pool packages with their closure; writes Packages + explicit.list and checks the set offline in a fresh base container (--network none). - offline-apt-install.sh: new; also used by that check (--simulate). - build-usb.sh: ships Packages, explicit.list and the script; refuses a set without them. - tests/vm: pxetest-client-nosb (Secure Boot off, e1000e, SATA), one client at a time.
2026-10-15 10:59:30 -04:00
parent c1a5fb3e02
commit bcd91b488a
13 changed files with 1034 additions and 671 deletions
--- a/tests/vm/README.md
+++ b/tests/vm/README.md
@@ -10,7 +10,8 @@ a laptop, without root and without touching a real network.
 | `pxetest-imaging` | libvirt network, bridge `virbr-pxetest`. No `<forward>`, no `<ip>`: the host has no address on it and libvirt runs no DHCP there. Only VM taps are on the bridge. |
 | `default`         | libvirt NAT, 192.168.122.0/24. Stands in for the GE/corp network.    |
 | `pxetest-server`  | 2 vCPU, 4 GiB, q35 + OVMF Secure Boot (MS keys), 60G thin qcow2 (`vda`, boot 1), the USB image as a USB disk (`sda`, boot 2). |
-| `pxetest-client`  | 2 vCPU, 2 GiB, OVMF Secure Boot (MS keys), 1G blank disk, network boot first (UEFI IPv4 PXE, virtio-net, OVMF driver, no option ROM). |
+| `pxetest-client`  | 2 vCPU, 2 GiB, OVMF Secure Boot (MS keys), 1G blank disk, network boot first (UEFI IPv4 PXE, virtio-net, OVMF driver, no option ROM). Proves shim -> iPXE -> menu -> WinPE under Secure Boot; WinPE has no virtio driver, so it stops at "Waiting for network". |
+| `pxetest-client-nosb` | 2 vCPU, 2 GiB, OVMF without Secure Boot, e1000e NIC, 64G thin SATA disk. WinPE has in-box drivers for both, so startnet gets a network and maps the shares. |

 NIC MACs (fixed):

@@ -19,6 +20,7 @@ NIC MACs (fixed):
 | server | GE      | `52:54:00:9e:10:01` | `default` NAT     | `corp_nic_mac`    |
 | server | imaging | `52:54:00:9e:10:02` | `pxetest-imaging` | `pxe_nic_mac`     |
 | client | -       | `52:54:00:9e:10:03` | `pxetest-imaging` | -                 |
+| client-nosb | -  | `52:54:00:9e:10:04` | `pxetest-imaging` | -                 |

 `build-image.sh` writes `$PXETEST_CACHE/site-vars.yml` with these two
 MACs (`pxe_nic_mac`, `corp_nic_mac`) and passes it to `build-usb.sh
@@ -40,14 +42,19 @@ IPv4 address or a non-tap bridge port.

    export PXETEST_CACHE=~/.cache/pxetest               # ISO + image (about 10 GB)
    export PXETEST_PAYLOAD_ROOT=/home/camp/projects/pxe  # checkout holding offline-packages/, boot-files/, ...
+    export UBUNTU_ISO_NAME=ubuntu-24.04.3-live-server-amd64.iso  # optional; default 24.04.5

-    tests/vm/fetch-iso.sh        # Ubuntu 24.04.5 live-server ISO, SHA256SUMS-verified
+    tests/vm/fetch-iso.sh        # the live-server ISO, SHA256SUMS-verified
+    scripts/download-packages.sh --iso "$PXETEST_CACHE/$UBUNTU_ISO_NAME" "$PXETEST_PAYLOAD_ROOT/offline-packages"
+    scripts/check-updates.py --debs "$PXETEST_PAYLOAD_ROOT/offline-packages"
    tests/vm/build-image.sh      # build-usb.sh --image, upload as volume pxetest-usb.img
    tests/vm/create.sh           # network, volumes, both VMs (prints the MAC pins)
    tests/vm/start.sh server     # autoinstall, reboot, first-boot playbook
    tests/vm/status.sh --shot /tmp/pxetest-shots   # states, leases, screenshots
-    tests/vm/start.sh client     # once the server playbook is done
-    tests/vm/stop.sh             # ACPI shutdown both, keep them defined
+    tests/vm/start.sh client     # once the server playbook is done (Secure Boot client)
+    tests/vm/stop.sh client
+    tests/vm/start.sh client2    # Secure Boot off, e1000e: WinPE maps the shares
+    tests/vm/stop.sh             # ACPI shutdown all, keep them defined
    tests/vm/destroy.sh          # remove every pxetest-* object

 `build-image.sh` skips enrollment/, bios-staging/, drivers-staging/ and
@@ -62,5 +69,10 @@ to `/var/log/first-boot.log`. Once the GE NIC takes a DHCP lease on the NAT
 network, `status.sh` shows its address and `ssh pxe@<address>` works from
 the host.

-Memory: server 4 GiB + client 2 GiB. Keep other VMs (win11) off while
+The offline package set must be built against the SAME ISO the image is
+built from (download-packages.sh resolves against that ISO's base). For a
+fresh server disk: `tests/vm/destroy.sh`, then build-image.sh, create.sh,
+start.sh server.
+
+Memory: server 4 GiB + one client 2 GiB (start.sh refuses a second client). Keep other VMs (win11) off while
 both run, and stop them when done.
--- a/tests/vm/common.sh
+++ b/tests/vm/common.sh
@@ -47,6 +47,11 @@ CLIENT_MEM_MIB="${CLIENT_MEM_MIB:-2048}"
 SERVER_GE_MAC="52:54:00:9e:10:01"       # on 'default' NAT (GE/corp side)
 SERVER_IMAGING_MAC="52:54:00:9e:10:02"  # on pxetest-imaging (imaging side)
 CLIENT_MAC="52:54:00:9e:10:03"          # on pxetest-imaging only
+# Second client: Secure Boot OFF, e1000e NIC and SATA disk (in-box WinPE
+# drivers), so WinPE gets a network and startnet can map the shares.
+CLIENT2_VM="pxetest-client-nosb"
+CLIENT2_DISK_VOL="pxetest-client-nosb.qcow2"
+CLIENT2_MAC="52:54:00:9e:10:04"         # on pxetest-imaging only

 log() { printf '[pxetest] %s\n' "$*"; }
 die() { printf '[pxetest] ERROR: %s\n' "$*" >&2; exit 1; }
--- a/tests/vm/create.sh
+++ b/tests/vm/create.sh
@@ -43,6 +43,10 @@ if ! vol_exists "$SERVER_DISK_VOL"; then
    virsh vol-create-as "$POOL" "$SERVER_DISK_VOL" "$SERVER_DISK_SIZE" --format qcow2 >/dev/null
    log "created $SERVER_DISK_VOL ($SERVER_DISK_SIZE thin qcow2)"
 fi
+if ! vol_exists "$CLIENT2_DISK_VOL"; then
+    virsh vol-create-as "$POOL" "$CLIENT2_DISK_VOL" 64G --format qcow2 >/dev/null
+    log "created $CLIENT2_DISK_VOL (64G thin, blank)"
+fi
 if ! vol_exists "$CLIENT_DISK_VOL"; then
    virsh vol-create-as "$POOL" "$CLIENT_DISK_VOL" 1G --format qcow2 >/dev/null
    log "created $CLIENT_DISK_VOL (1G blank)"
@@ -157,8 +161,49 @@ XML
    log "defined $CLIENT_VM"
 fi

+# --- second client: Secure Boot off, e1000e + SATA (WinPE in-box drivers) ---
+if ! dom_exists "$CLIENT2_VM"; then
+    cat >"$TMP/client2.xml" <<XML
+<domain type='kvm'>
+  <name>$CLIENT2_VM</name>
+  <memory unit='MiB'>$CLIENT_MEM_MIB</memory>
+  <vcpu>2</vcpu>
+  <os firmware='efi'>
+    <type arch='x86_64' machine='q35'>hvm</type>
+    <firmware>
+      <feature enabled='no' name='secure-boot'/>
+    </firmware>
+  </os>
+  <features><acpi/><apic/></features>
+  <cpu mode='host-passthrough'/>
+  <clock offset='localtime'/>
+  <devices>
+    <interface type='network'>
+      <source network='$IMAGING_NET'/>
+      <mac address='$CLIENT2_MAC'/>
+      <model type='e1000e'/>
+      <boot order='1'/>
+    </interface>
+    <disk type='file' device='disk'>
+      <driver name='qemu' type='qcow2'/>
+      <source file='$POOL_DIR/$CLIENT2_DISK_VOL'/>
+      <target dev='sda' bus='sata'/>
+      <boot order='2'/>
+    </disk>
+    <serial type='pty'/>
+    <console type='pty'/>
+    <graphics type='vnc' listen='127.0.0.1' autoport='yes'/>
+    <video><model type='vga'/></video>
+    <memballoon model='virtio'/>
+  </devices>
+</domain>
+XML
+    virsh define "$TMP/client2.xml" >/dev/null
+    log "defined $CLIENT2_VM"
+fi
+
 # Never attach a pxetest VM anywhere but default NAT / pxetest-imaging.
-for vm in "$SERVER_VM" "$CLIENT_VM"; do
+for vm in "$SERVER_VM" "$CLIENT_VM" "$CLIENT2_VM"; do
    while read -r _ type src _ _; do
        case "$type:$src" in
            network:"$GE_NET") [ "$vm" = "$SERVER_VM" ] || die "$vm is on $GE_NET" ;;
--- a/tests/vm/destroy.sh
+++ b/tests/vm/destroy.sh
@@ -5,13 +5,13 @@
 # Usage: tests/vm/destroy.sh
 source "$(dirname "$0")/common.sh"

-for vm in "$CLIENT_VM" "$SERVER_VM"; do
+for vm in "$CLIENT_VM" "$CLIENT2_VM" "$SERVER_VM"; do
    dom_exists "$vm" || continue
    [ "$(dom_state "$vm")" = "shut off" ] || virsh destroy "$vm" >/dev/null
    virsh undefine "$vm" --nvram >/dev/null
    log "undefined $vm"
 done
-for v in "$SERVER_DISK_VOL" "$USB_VOL" "$CLIENT_DISK_VOL"; do
+for v in "$SERVER_DISK_VOL" "$USB_VOL" "$CLIENT_DISK_VOL" "$CLIENT2_DISK_VOL"; do
    vol_exists "$v" || continue
    virsh vol-delete --pool "$POOL" "$v" >/dev/null
    log "deleted volume $v"
--- a/tests/vm/start.sh
+++ b/tests/vm/start.sh
@@ -1,16 +1,19 @@
 #!/bin/bash
 # Start pxetest VMs. Idempotent (a running VM is left alone).
-# Usage: tests/vm/start.sh [server|client|all]   (default: server)
-# Start the client only once the server's playbook has finished; both
-# together use 6 GiB.
+# Usage: tests/vm/start.sh [server|client|client2|all]   (default: server)
+#   client  = Secure Boot client (virtio NIC: reaches the iPXE menu and WinPE)
+#   client2 = Secure Boot OFF client, e1000e NIC (WinPE gets a network)
+# Start a client only once the server's playbook has finished, and only one
+# client at a time: server + one client = 6 GiB.
 source "$(dirname "$0")/common.sh"

 which="${1:-server}"
 case "$which" in
    server) vms=("$SERVER_VM") ;;
    client) vms=("$CLIENT_VM") ;;
+    client2|nosb) vms=("$CLIENT2_VM") ;;
    all) vms=("$SERVER_VM" "$CLIENT_VM") ;;
-    *) die "usage: $0 [server|client|all]" ;;
+    *) die "usage: $0 [server|client|client2|all]" ;;
 esac

 net_exists "$IMAGING_NET" || die "run create.sh first"
@@ -19,6 +22,15 @@ assert_imaging_isolated

 for vm in "${vms[@]}"; do
    dom_exists "$vm" || die "$vm not defined; run create.sh"
+    # RAM: one client at a time.
+    case "$vm" in
+        "$CLIENT_VM") other="$CLIENT2_VM" ;;
+        "$CLIENT2_VM") other="$CLIENT_VM" ;;
+        *) other="" ;;
+    esac
+    if [ -n "$other" ] && [ "$(dom_state "$other")" = "running" ]; then
+        die "$other is running; stop it first (tests/vm/stop.sh) to stay within RAM"
+    fi
    if [ "$vm" = "$SERVER_VM" ]; then
        vol_exists "$USB_VOL" || die "no $USB_VOL; run build-image.sh"
    fi
--- a/tests/vm/status.sh
+++ b/tests/vm/status.sh
@@ -12,14 +12,14 @@ echo "network $IMAGING_NET: $n"
 if net_active "$IMAGING_NET"; then
    (assert_imaging_isolated && echo "  isolation: OK (no forward, no ip, no host IPv4, only vnet ports)")
 fi
-for v in "$SERVER_DISK_VOL" "$USB_VOL" "$CLIENT_DISK_VOL"; do
+for v in "$SERVER_DISK_VOL" "$USB_VOL" "$CLIENT_DISK_VOL" "$CLIENT2_DISK_VOL"; do
    if vol_exists "$v"; then
        echo "volume $v: $(virsh vol-info --pool "$POOL" "$v" | awk '/Capacity|Allocation/{printf "%s %s %s  ", $1, $2, $3}')"
    else
        echo "volume $v: absent"
    fi
 done
-for vm in "$SERVER_VM" "$CLIENT_VM"; do
+for vm in "$SERVER_VM" "$CLIENT_VM" "$CLIENT2_VM"; do
    if dom_exists "$vm"; then
        echo "domain $vm: $(dom_state "$vm")"
        virsh domiflist "$vm" | tail -n +3 | grep -v '^$' | sed 's/^/    /'
--- a/tests/vm/stop.sh
+++ b/tests/vm/stop.sh
@@ -1,15 +1,16 @@
 #!/bin/bash
 # Shut pxetest VMs down (ACPI, then force after a timeout). Keeps them
 # defined. Idempotent.
-# Usage: tests/vm/stop.sh [server|client|all]   (default: all)
+# Usage: tests/vm/stop.sh [server|client|client2|all]   (default: all)
 source "$(dirname "$0")/common.sh"

 which="${1:-all}"
 case "$which" in
    server) vms=("$SERVER_VM") ;;
    client) vms=("$CLIENT_VM") ;;
-    all) vms=("$CLIENT_VM" "$SERVER_VM") ;;
-    *) die "usage: $0 [server|client|all]" ;;
+    client2|nosb) vms=("$CLIENT2_VM") ;;
+    all) vms=("$CLIENT_VM" "$CLIENT2_VM" "$SERVER_VM") ;;
+    *) die "usage: $0 [server|client|client2|all]" ;;
 esac
 for vm in "${vms[@]}"; do
    dom_exists "$vm" || continue