NICs: pick the imaging port at install, GE can go into any other port

Sites no longer need either MAC in site-vars.

pxe_nic_mac empty: before anything changes, the playbook runs
pxe-pick-nic --auto. It takes the one wired port with a link, accepts it
only if pxe-nic-check calls it isolated (no LLDP/CDP/802.1X, no other
DHCP server, at most 10 hosts heard in 65 s), and pins it. The choice is
saved in /etc/pxe-server/nic-pick.conf and never re-guessed at boot.
When it cannot tell (no link, two links, not isolated) nothing changes,
and the console banner says why and to run sudo pxe-pick-nic: pull the
imaging cable and plug it back in, and the port that blinked is the
imaging one. Both helpers are installed before the pick, so that works
after a refused first boot. The site's corp domains and allowed
switches are passed to the check.

corp_nic_mac empty: every other wired port (en*, eth*) is a catch-all
DHCP client, so GE can be plugged in at install or later.
- dnsmasq stays pinned with interface=pxe0. A new check refuses any
  dhcp-range outside the imaging subnet, and proxy ranges.
- The catch-all ports ignore offers from 172.16.9.1.
- UFW allows SSH and 9009 by source on any interface, and keeps those
  rules when the old any-interface rules are removed.
- pxe_nic_name must not look like en*/eth* in this mode.

pxe-netplan-apply.sh takes 3 or 5 args. Without a pinned GE NIC it:
- renames stray names (the old pxe0 after a re-pick, a leftover corp0)
  to enx<mac>, so the catch-all takes them, and restores them on revert;
- checks that every NIC that had a DHCP address before has one after,
  tracked by MAC.

With both MACs set (the WJ server), netplan, the dnsmasq pin and UFW
render byte-for-byte as before.

Tested in the pxetest VMs:
- Fresh install with no MACs and the GE link down: the imaging port was
  picked and pinned as pxe0 with 172.16.9.1, and dnsmasq bound DHCP to
  pxe0 only. GE plugged in afterwards got a lease, SSH worked, and the
  guard reported pxe0 isolated.
- --auto refuses with two links.
- The interactive pick works by pulling and replugging the link.
- A 3-arg re-apply is UNCHANGED.

Unit tests (fake sysfs) cover the re-pick and pinned-to-catch-all renames.
This commit is contained in:
cproudlock
2026-10-16 15:56:47 -04:00
parent 9af1c6d89a
commit b11e678297
12 changed files with 774 additions and 108 deletions

View File

@@ -2,18 +2,26 @@
# Build the installer USB as a disk image FILE with build-usb.sh image mode,
# then upload it into the libvirt 'default' pool as volume pxetest-usb.img.
#
# Usage: tests/vm/build-image.sh [--with-enrollment] [--with-boot-tools] [--size SIZE]
# Default skips enrollment/, bios-staging/, drivers-staging/ and boot-tools/
# to keep the image small; SIZE defaults to "auto".
# Usage: tests/vm/build-image.sh [--with-enrollment] [--with-boot-tools]
# [--with-site-content] [--pick-nic] [--size SIZE]
# Default skips enrollment/, bios-staging/, drivers-staging/, boot-tools/
# and site-content/ to keep the image small; SIZE defaults to "auto".
# --pick-nic leaves both MACs out of site-vars: the first boot picks the
# imaging NIC by its link and every other port is a DHCP client (start the
# server with its GE NIC unplugged: virsh domif-setlink pxetest-server
# 52:54:00:9e:10:01 down --config).
# Env: PXETEST_PAYLOAD_ROOT (where offline-packages/, boot-files/... live)
source "$(dirname "$0")/common.sh"
SIZE=auto
SKIPS=(--skip-enrollment --skip-bios --skip-drivers --skip-boot-tools)
SKIPS=(--skip-enrollment --skip-bios --skip-drivers --skip-boot-tools --skip-site-content)
PICK_NIC=0
while [ $# -gt 0 ]; do
case "$1" in
--with-enrollment) SKIPS=("${SKIPS[@]/--skip-enrollment}"); shift ;;
--with-boot-tools) SKIPS=("${SKIPS[@]/--skip-boot-tools}"); shift ;;
--with-site-content) SKIPS=("${SKIPS[@]/--skip-site-content}"); shift ;;
--pick-nic) PICK_NIC=1; shift ;;
--size) SIZE="$2"; shift 2 ;;
*) die "unknown option $1" ;;
esac
@@ -43,13 +51,24 @@ fi
# Site vars for the VM "site": pin the NICs by the fixed MACs from common.sh.
SITE_VARS="$PXETEST_CACHE/site-vars.yml"
cat >"$SITE_VARS" <<YML
if [ "$PICK_NIC" -eq 1 ]; then
cat >"$SITE_VARS" <<YML
---
# Generated by tests/vm/build-image.sh --pick-nic for the pxetest VMs: no
# MACs, the first boot picks the imaging NIC by its link.
pxe_nic_mac: ""
corp_nic_mac: ""
YML
log "site vars: no MACs (first boot picks the imaging NIC)"
else
cat >"$SITE_VARS" <<YML
---
# Generated by tests/vm/build-image.sh for the pxetest VMs.
pxe_nic_mac: "$SERVER_IMAGING_MAC" # pxetest-imaging (isolated)
corp_nic_mac: "$SERVER_GE_MAC" # libvirt default NAT ("GE/corp")
YML
log "site vars: pxe_nic_mac=$SERVER_IMAGING_MAC corp_nic_mac=$SERVER_GE_MAC"
log "site vars: pxe_nic_mac=$SERVER_IMAGING_MAC corp_nic_mac=$SERVER_GE_MAC"
fi
"$REPO_ROOT/scripts/build-usb.sh" --yes --payload-root "$PXETEST_PAYLOAD_ROOT" \
--site-vars "$SITE_VARS" \