ge-aerospace/pxe-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

NICs: pick the imaging port at install, GE can go into any other port

Browse Source 
Sites no longer need either MAC in site-vars.

pxe_nic_mac empty: before anything changes, the playbook runs
pxe-pick-nic --auto. It takes the one wired port with a link, accepts it
only if pxe-nic-check calls it isolated (no LLDP/CDP/802.1X, no other
DHCP server, at most 10 hosts heard in 65 s), and pins it. The choice is
saved in /etc/pxe-server/nic-pick.conf and never re-guessed at boot.
When it cannot tell (no link, two links, not isolated) nothing changes,
and the console banner says why and to run sudo pxe-pick-nic: pull the
imaging cable and plug it back in, and the port that blinked is the
imaging one. Both helpers are installed before the pick, so that works
after a refused first boot. The site's corp domains and allowed
switches are passed to the check.

corp_nic_mac empty: every other wired port (en*, eth*) is a catch-all
DHCP client, so GE can be plugged in at install or later.
- dnsmasq stays pinned with interface=pxe0. A new check refuses any
  dhcp-range outside the imaging subnet, and proxy ranges.
- The catch-all ports ignore offers from 172.16.9.1.
- UFW allows SSH and 9009 by source on any interface, and keeps those
  rules when the old any-interface rules are removed.
- pxe_nic_name must not look like en*/eth* in this mode.

pxe-netplan-apply.sh takes 3 or 5 args. Without a pinned GE NIC it:
- renames stray names (the old pxe0 after a re-pick, a leftover corp0)
  to enx<mac>, so the catch-all takes them, and restores them on revert;
- checks that every NIC that had a DHCP address before has one after,
  tracked by MAC.

With both MACs set (the WJ server), netplan, the dnsmasq pin and UFW
render byte-for-byte as before.

Tested in the pxetest VMs:
- Fresh install with no MACs and the GE link down: the imaging port was
  picked and pinned as pxe0 with 172.16.9.1, and dnsmasq bound DHCP to
  pxe0 only. GE plugged in afterwards got a lease, SSH worked, and the
  guard reported pxe0 isolated.
- --auto refuses with two links.
- The interactive pick works by pulling and replugging the link.
- A 3-arg re-apply is UNCHANGED.

Unit tests (fake sysfs) cover the re-pick and pinned-to-catch-all renames.
This commit is contained in:
cproudlock
2026-10-16 15:56:47 -04:00
parent 9af1c6d89a
commit b11e678297
12 changed files with 774 additions and 108 deletions
Download Patch File Download Diff File Expand all files Collapse all files

152
playbook/pxe-server-helpers/pxe-netplan-apply.sh
View File

@@ -1,12 +1,17 @@
#!/bin/bash
# pxe-netplan-apply.sh - install the two-NIC netplan config without ever
# leaving the box unreachable on the corporate NIC.
# pxe-netplan-apply.sh - install the PXE server's netplan config without
# ever leaving the box unreachable on the corporate side.
#
# Usage:
# pxe-netplan-apply.sh STAGED_YAML PXE_NAME PXE_MAC CORP_NAME CORP_MAC
# pxe-netplan-apply.sh --preflight STAGED_YAML PXE_NAME PXE_MAC CORP_NAME CORP_MAC
# pxe-netplan-apply.sh STAGED_YAML PXE_NAME PXE_MAC [CORP_NAME CORP_MAC]
# pxe-netplan-apply.sh --preflight STAGED_YAML PXE_NAME PXE_MAC [CORP_NAME CORP_MAC]
# pxe-netplan-apply.sh --revert BACKUP_DIR (the auto-revert timer)
#
# Without CORP_NAME/CORP_MAC (corp_nic_mac empty in the playbook) only the
# imaging NIC is pinned and renamed; the other wired ports are catch-all
# DHCP clients that keep their names. Every pre-flight refusal about the
# imaging NIC applies the same.
#
# Env (set by pxe_server_setup.yml):
# PXE_SUBNET imaging LAN, e.g. 172.16.9.0/24 (SSH peers inside it may
# legitimately come in on the imaging NIC)
@@ -128,14 +133,16 @@ ssh_peers() {
preflight() {
local pif cif peer dev bad=0
pif=$(iface_of "$PXE_MAC")
cif=$(iface_of "$CORP_MAC")
if [ -z "$pif" ]; then
log "REFUSED: no physical NIC has pxe_nic_mac $PXE_MAC"
return 1
fi
if [ -z "$cif" ]; then
log "REFUSED: no physical NIC has corp_nic_mac $CORP_MAC"
return 1
if [ -n "$CORP_MAC" ]; then
cif=$(iface_of "$CORP_MAC")
if [ -z "$cif" ]; then
log "REFUSED: no physical NIC has corp_nic_mac $CORP_MAC"
return 1
fi
fi
if default_route_devs | grep -qxF "$pif"; then
log "REFUSED: $pif (pxe_nic_mac $PXE_MAC) holds the default route"
@@ -156,8 +163,8 @@ preflight() {
bad=1
done < <(ssh_peers)
if [ "$bad" -ne 0 ]; then
log "The NIC named by pxe_nic_mac looks like the GE corporate one: pxe_nic_mac"
log "and corp_nic_mac are probably swapped, or the cables are. It would become"
log "The NIC named by pxe_nic_mac looks like the GE corporate one: the MACs are"
log "probably swapped, or the cables are. It would become"
log "static ${PXE_SUBNET:-imaging} with no gateway and cut this box off. Nothing was changed."
log "NICs now: $(for d in "$SYSFS_NET"/*; do [ -e "$d/device" ] && printf '%s=%s ' "$(basename "$d")" "$(cat "$d/address" 2>/dev/null)"; done)"
return 1
@@ -181,32 +188,38 @@ link_rename() {
return 1
}
# set_names MAC1 NAME1 MAC2 NAME2: give each MAC its name. A NIC that must
# set_names MAC1 NAME1 [MAC2 NAME2]: give each MAC its name. A NIC that must
# change goes through a temporary name first, so a swap (pxe0 <-> corp0)
# works in both directions. Temporary names carry this run's PID, so they
# never collide with a name left over from an earlier (failed) pass.
# never collide with a name left over from an earlier (failed) pass. A pair
# with an empty MAC or name is skipped (no pinned GE NIC).
set_names() {
local -a macs=("$1" "$3") want=("$2" "$4") cur=()
local -a macs=() want=() cur=()
local i
while [ $# -ge 2 ]; do
if [ -n "$1" ] && [ -n "$2" ]; then
macs+=("$1"); want+=("$2")
fi
shift 2
done
RENAME_SEQ=$(( ${RENAME_SEQ:-0} + 1 ))
for i in 0 1; do
[ -n "${want[i]}" ] || return 0
for i in "${!macs[@]}"; do
cur[i]=$(iface_of "${macs[i]}")
if [ -z "${cur[i]}" ]; then
log "ERROR: no NIC with MAC ${macs[i]}"
return 1
fi
done
for i in 0 1; do
for i in "${!macs[@]}"; do
[ "${cur[i]}" = "${want[i]}" ] && continue
link_rename "${cur[i]}" "pxt$$-$RENAME_SEQ$i" || return 1
cur[i]=pxt$$-$RENAME_SEQ$i
done
for i in 0 1; do
for i in "${!macs[@]}"; do
[ "${cur[i]}" = "${want[i]}" ] && continue
link_rename "${cur[i]}" "${want[i]}" || return 1
done
for i in 0 1; do
for i in "${!macs[@]}"; do
if [ "$(iface_of "${macs[i]}")" != "${want[i]}" ]; then
log "ERROR: MAC ${macs[i]} is not named ${want[i]} after renaming"
return 1
@@ -216,8 +229,49 @@ set_names() {
}
names_ok() {
[ "$(iface_of "$PXE_MAC")" = "$PXE_NAME" ] &&
[ "$(iface_of "$PXE_MAC")" = "$PXE_NAME" ] || return 1
if [ -n "$CORP_MAC" ]; then
[ "$(iface_of "$CORP_MAC")" = "$CORP_NAME" ]
else
[ -z "$(stray_names)" ]
fi
}
# No pinned GE NIC: "MAC NAME" for every other physical NIC whose name the
# catch-all (en*, eth*) would not match - the previous imaging NIC still
# called pxe0 after a re-pick, or corp0 from a pinned config. Each gets the
# kernel-style MAC name enx<mac>, so it becomes a catch-all DHCP client and
# pxe0 is free for the new imaging NIC.
stray_names() {
local d n m
[ -z "$CORP_MAC" ] || return 0
for d in "$SYSFS_NET"/*; do
[ -e "$d/device" ] || continue
n=${d##*/}
m=$(cat "$d/address" 2>/dev/null)
[ "$m" = "$PXE_MAC" ] && continue
[ "$(cat "$d/type" 2>/dev/null)" = 1 ] || continue
[ -e "$d/wireless" ] || [ -e "$d/phy80211" ] || [ -e "$d/bridge" ] && continue
case "$n" in en*|eth*) continue ;; esac
echo "$m enx${m//:/}"
done
return 0
}
# MACs of the physical NICs other than the imaging one that hold a DHCP
# (dynamic) IPv4: the GE side. They must have one again after the apply,
# or an admin on the GE side may have been cut off. Static addresses (the
# previous imaging NIC's 172.16.9.1) do not count. By MAC: names change.
other_v4_macs() {
local d n m
for d in "$SYSFS_NET"/*; do
[ -e "$d/device" ] || continue
n=${d##*/}
m=$(cat "$d/address" 2>/dev/null)
[ "$m" = "$PXE_MAC" ] && continue
ip -4 -o addr show dev "$n" scope global dynamic 2>/dev/null | grep -q ' inet ' && echo "$m"
done
return 0
}
# --- backup / restore --------------------------------------------------------
@@ -239,6 +293,7 @@ write_state() {
echo "ORIG_PXE_IF=$ORIG_PXE_IF"
echo "CORP_MAC=$CORP_MAC"
echo "ORIG_CORP_IF=$ORIG_CORP_IF"
echo "EXTRA=$EXTRA"
} >"$BACKUP.state"
}
@@ -249,6 +304,7 @@ read_state() {
NETPLAN_DIR) [[ "$v" =~ ^/[A-Za-z0-9/._-]*$ ]] && NETPLAN_DIR=$v ;;
PXE_MAC|CORP_MAC) [[ "$v" =~ ^([0-9a-f]{2}:){5}[0-9a-f]{2}$ ]] && printf -v "$k" '%s' "$v" ;;
ORIG_PXE_IF|ORIG_CORP_IF) [[ "$v" =~ ^[A-Za-z0-9._-]{0,15}$ ]] && printf -v "$k" '%s' "$v" ;;
EXTRA) [[ "$v" =~ ^(([0-9a-f]{2}:){5}[0-9a-f]{2}=[A-Za-z0-9._-]{1,15},?)*$ ]] && EXTRA=$v ;;
esac
done <"$BACKUP.state"
TARGET=$NETPLAN_DIR/50-cloud-init.yaml
@@ -268,8 +324,11 @@ restore() {
log "restoring the previous config from $BACKUP"
find "$NETPLAN_DIR" -maxdepth 1 -type f \( -name '*.yaml' -o -name '*.yml' \) -delete
cp -a "$BACKUP/." "$NETPLAN_DIR/"
if [ -n "${ORIG_PXE_IF:-}" ] && [ -n "${ORIG_CORP_IF:-}" ]; then
set_names "$PXE_MAC" "$ORIG_PXE_IF" "$CORP_MAC" "$ORIG_CORP_IF" ||
if [ -n "${ORIG_PXE_IF:-}" ]; then
local -a back=()
local pair
for pair in ${EXTRA//,/ }; do back+=("${pair%%=*}" "${pair#*=}"); done
set_names "$PXE_MAC" "$ORIG_PXE_IF" "$CORP_MAC" "${ORIG_CORP_IF:-}" "${back[@]}" ||
{ log "ERROR: could not put the previous NIC names back"; rc=1; }
fi
netplan generate && netplan apply || { log "ERROR: netplan apply of the restored config failed"; rc=1; }
@@ -300,7 +359,7 @@ case "${1:-}" in
log "revert: $BACKUP already committed or reverted; nothing to do"
exit 0
fi
PXE_MAC="" CORP_MAC="" ORIG_PXE_IF="" ORIG_CORP_IF=""
PXE_MAC="" CORP_MAC="" ORIG_PXE_IF="" ORIG_CORP_IF="" EXTRA=""
read_state
log "revert: the apply was not confirmed within its window"
restore
@@ -308,15 +367,16 @@ case "${1:-}" in
;;
esac
if [ $# -ne 5 ]; then
echo "usage: $0 [--preflight] STAGED_YAML PXE_NAME PXE_MAC CORP_NAME CORP_MAC" >&2
if [ $# -ne 3 ] && [ $# -ne 5 ]; then
echo "usage: $0 [--preflight] STAGED_YAML PXE_NAME PXE_MAC [CORP_NAME CORP_MAC]" >&2
exit 2
fi
STAGED=$1
PXE_NAME=$2
PXE_MAC=${3,,}
CORP_NAME=$4
CORP_MAC=${5,,}
CORP_NAME=${4:-}
CORP_MAC=${5:-}
CORP_MAC=${CORP_MAC,,}
# 1. validate in a scratch root
SCRATCH=$(mktemp -d)
@@ -330,7 +390,7 @@ fi
# 2. already in place?
if cmp -s "$STAGED" "$TARGET" && [ -z "$(other_files)" ] && names_ok; then
log "netplan already in place, $PXE_NAME and $CORP_NAME present"
log "netplan already in place, $PXE_NAME${CORP_NAME:+ and $CORP_NAME} present"
echo UNCHANGED
exit 0
fi
@@ -350,14 +410,26 @@ flock -w 60 9 || { log "ERROR: another pxe-netplan-apply holds $LOCK_FILE; nothi
# 4. back up, record names, arm the auto-revert
ORIG_PXE_IF=$(iface_of "$PXE_MAC")
ORIG_CORP_IF=$(iface_of "$CORP_MAC")
ORIG_CORP_IF=""
[ -n "$CORP_MAC" ] && ORIG_CORP_IF=$(iface_of "$CORP_MAC")
CORP_HAD_V4=0
has_v4 "$ORIG_CORP_IF" && CORP_HAD_V4=1
[ -n "$ORIG_CORP_IF" ] && has_v4 "$ORIG_CORP_IF" && CORP_HAD_V4=1
OTHERS_V4=""
[ -z "$CORP_MAC" ] && OTHERS_V4=$(other_v4_macs)
# NICs to move off a name the catch-all would not match, recorded with
# their current names for the revert ("mac=name,...").
EXTRA=""
FREE=()
while read -r m newname; do
[ -n "$m" ] || continue
EXTRA="${EXTRA:+$EXTRA,}$m=$(iface_of "$m")"
FREE+=("$m" "$newname")
done < <(stray_names)
BACKUP=$BACKUP_ROOT/netplan-$(date +%Y%m%d-%H%M%S)
mkdir -p "$BACKUP"
cp -a "$NETPLAN_DIR/." "$BACKUP/"
write_state
log "previous config saved in $BACKUP (NICs: $ORIG_PXE_IF, $ORIG_CORP_IF)"
log "previous config saved in $BACKUP (NICs: $ORIG_PXE_IF${ORIG_CORP_IF:+, $ORIG_CORP_IF})"
REVERT_UNIT=""
if [ "$REVERT_AFTER" -gt 0 ] 2>/dev/null && command -v systemd-run >/dev/null; then
@@ -389,20 +461,32 @@ netplan generate || fail "netplan generate failed"
# 6. stable names now, so dnsmasq/UFW/Samba (which use them) work before
# the next reboot. Does not depend on the netplan version renaming.
set_names "$PXE_MAC" "$PXE_NAME" "$CORP_MAC" "$CORP_NAME" || fail "NIC rename failed"
[ ${#FREE[@]} -gt 0 ] && log "freeing names for the catch-all: $EXTRA"
set_names "$PXE_MAC" "$PXE_NAME" "$CORP_MAC" "$CORP_NAME" "${FREE[@]}" || fail "NIC rename failed"
# 7. apply
netplan apply || fail "netplan apply failed"
# 8. check
names_ok || fail "$PXE_NAME / $CORP_NAME are not on MACs $PXE_MAC / $CORP_MAC after apply"
names_ok || fail "$PXE_NAME${CORP_NAME:+ / $CORP_NAME} not on MAC $PXE_MAC${CORP_MAC:+ / $CORP_MAC} after apply"
ok=0
for _ in $(seq 1 "$PXE_WAIT"); do
if has_v4 "$PXE_NAME"; then ok=1; break; fi
sleep 1
done
[ "$ok" -eq 1 ] || fail "$PXE_NAME has no IPv4 address after ${PXE_WAIT}s"
if [ "$CORP_HAD_V4" -eq 1 ]; then
for m in $OTHERS_V4; do
ok=0
for _ in $(seq 1 "$CORP_WAIT"); do
n=$(iface_of "$m")
if [ -n "$n" ] && has_v4 "$n"; then ok=1; break; fi
sleep 1
done
[ "$ok" -eq 1 ] || fail "the NIC with MAC $m had a DHCP IPv4 before and has none after ${CORP_WAIT}s"
done
if [ -z "$CORP_MAC" ]; then
:
elif [ "$CORP_HAD_V4" -eq 1 ]; then
ok=0
for _ in $(seq 1 "$CORP_WAIT"); do
if has_v4 "$CORP_NAME"; then ok=1; break; fi
@@ -427,6 +511,6 @@ touch "$BACKUP.committed"
cancel_revert
trap - INT TERM
ip -br addr show dev "$PXE_NAME" 2>/dev/null
ip -br addr show dev "$CORP_NAME" 2>/dev/null
[ -n "$CORP_NAME" ] && ip -br addr show dev "$CORP_NAME" 2>/dev/null
echo CHANGED
exit 0