ge-aerospace/pxe-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

NICs: pick the imaging port at install, GE can go into any other port

Browse Source 
Sites no longer need either MAC in site-vars.

pxe_nic_mac empty: before anything changes, the playbook runs
pxe-pick-nic --auto. It takes the one wired port with a link, accepts it
only if pxe-nic-check calls it isolated (no LLDP/CDP/802.1X, no other
DHCP server, at most 10 hosts heard in 65 s), and pins it. The choice is
saved in /etc/pxe-server/nic-pick.conf and never re-guessed at boot.
When it cannot tell (no link, two links, not isolated) nothing changes,
and the console banner says why and to run sudo pxe-pick-nic: pull the
imaging cable and plug it back in, and the port that blinked is the
imaging one. Both helpers are installed before the pick, so that works
after a refused first boot. The site's corp domains and allowed
switches are passed to the check.

corp_nic_mac empty: every other wired port (en*, eth*) is a catch-all
DHCP client, so GE can be plugged in at install or later.
- dnsmasq stays pinned with interface=pxe0. A new check refuses any
  dhcp-range outside the imaging subnet, and proxy ranges.
- The catch-all ports ignore offers from 172.16.9.1.
- UFW allows SSH and 9009 by source on any interface, and keeps those
  rules when the old any-interface rules are removed.
- pxe_nic_name must not look like en*/eth* in this mode.

pxe-netplan-apply.sh takes 3 or 5 args. Without a pinned GE NIC it:
- renames stray names (the old pxe0 after a re-pick, a leftover corp0)
  to enx<mac>, so the catch-all takes them, and restores them on revert;
- checks that every NIC that had a DHCP address before has one after,
  tracked by MAC.

With both MACs set (the WJ server), netplan, the dnsmasq pin and UFW
render byte-for-byte as before.

Tested in the pxetest VMs:
- Fresh install with no MACs and the GE link down: the imaging port was
  picked and pinned as pxe0 with 172.16.9.1, and dnsmasq bound DHCP to
  pxe0 only. GE plugged in afterwards got a lease, SSH worked, and the
  guard reported pxe0 isolated.
- --auto refuses with two links.
- The interactive pick works by pulling and replugging the link.
- A 3-arg re-apply is UNCHANGED.

Unit tests (fake sysfs) cover the re-pick and pinned-to-catch-all renames.
This commit is contained in:
cproudlock
2026-10-16 15:56:47 -04:00
parent 9af1c6d89a
commit b11e678297
12 changed files with 774 additions and 108 deletions
Download Patch File Download Diff File Expand all files Collapse all files

152
playbook/pxe-server-helpers/pxe-netplan-apply.sh
View File

@@ -1,12 +1,17 @@
#!/bin/bash
# pxe-netplan-apply.sh - install the two-NIC netplan config without ever
# leaving the box unreachable on the corporate NIC.
# pxe-netplan-apply.sh - install the PXE server's netplan config without
# ever leaving the box unreachable on the corporate side.
#
# Usage:
# pxe-netplan-apply.sh STAGED_YAML PXE_NAME PXE_MAC CORP_NAME CORP_MAC
# pxe-netplan-apply.sh --preflight STAGED_YAML PXE_NAME PXE_MAC CORP_NAME CORP_MAC
# pxe-netplan-apply.sh STAGED_YAML PXE_NAME PXE_MAC [CORP_NAME CORP_MAC]
# pxe-netplan-apply.sh --preflight STAGED_YAML PXE_NAME PXE_MAC [CORP_NAME CORP_MAC]
# pxe-netplan-apply.sh --revert BACKUP_DIR (the auto-revert timer)
#
# Without CORP_NAME/CORP_MAC (corp_nic_mac empty in the playbook) only the
# imaging NIC is pinned and renamed; the other wired ports are catch-all
# DHCP clients that keep their names. Every pre-flight refusal about the
# imaging NIC applies the same.
#
# Env (set by pxe_server_setup.yml):
# PXE_SUBNET imaging LAN, e.g. 172.16.9.0/24 (SSH peers inside it may
# legitimately come in on the imaging NIC)
@@ -128,14 +133,16 @@ ssh_peers() {
preflight() {
local pif cif peer dev bad=0
pif=$(iface_of "$PXE_MAC")
cif=$(iface_of "$CORP_MAC")
if [ -z "$pif" ]; then
log "REFUSED: no physical NIC has pxe_nic_mac $PXE_MAC"
return 1
fi
if [ -z "$cif" ]; then
log "REFUSED: no physical NIC has corp_nic_mac $CORP_MAC"
return 1
if [ -n "$CORP_MAC" ]; then
cif=$(iface_of "$CORP_MAC")
if [ -z "$cif" ]; then
log "REFUSED: no physical NIC has corp_nic_mac $CORP_MAC"
return 1
fi
fi
if default_route_devs | grep -qxF "$pif"; then
log "REFUSED: $pif (pxe_nic_mac $PXE_MAC) holds the default route"
@@ -156,8 +163,8 @@ preflight() {
bad=1
done < <(ssh_peers)
if [ "$bad" -ne 0 ]; then
log "The NIC named by pxe_nic_mac looks like the GE corporate one: pxe_nic_mac"
log "and corp_nic_mac are probably swapped, or the cables are. It would become"
log "The NIC named by pxe_nic_mac looks like the GE corporate one: the MACs are"
log "probably swapped, or the cables are. It would become"
log "static ${PXE_SUBNET:-imaging} with no gateway and cut this box off. Nothing was changed."
log "NICs now: $(for d in "$SYSFS_NET"/*; do [ -e "$d/device" ] && printf '%s=%s ' "$(basename "$d")" "$(cat "$d/address" 2>/dev/null)"; done)"
return 1
@@ -181,32 +188,38 @@ link_rename() {
return 1
}
# set_names MAC1 NAME1 MAC2 NAME2: give each MAC its name. A NIC that must
# set_names MAC1 NAME1 [MAC2 NAME2]: give each MAC its name. A NIC that must
# change goes through a temporary name first, so a swap (pxe0 <-> corp0)
# works in both directions. Temporary names carry this run's PID, so they
# never collide with a name left over from an earlier (failed) pass.
# never collide with a name left over from an earlier (failed) pass. A pair
# with an empty MAC or name is skipped (no pinned GE NIC).
set_names() {
local -a macs=("$1" "$3") want=("$2" "$4") cur=()
local -a macs=() want=() cur=()
local i
while [ $# -ge 2 ]; do
if [ -n "$1" ] && [ -n "$2" ]; then
macs+=("$1"); want+=("$2")
fi
shift 2
done
RENAME_SEQ=$(( ${RENAME_SEQ:-0} + 1 ))
for i in 0 1; do
[ -n "${want[i]}" ] || return 0
for i in "${!macs[@]}"; do
cur[i]=$(iface_of "${macs[i]}")
if [ -z "${cur[i]}" ]; then
log "ERROR: no NIC with MAC ${macs[i]}"
return 1
fi
done
for i in 0 1; do
for i in "${!macs[@]}"; do
[ "${cur[i]}" = "${want[i]}" ] && continue
link_rename "${cur[i]}" "pxt$$-$RENAME_SEQ$i" || return 1
cur[i]=pxt$$-$RENAME_SEQ$i
done
for i in 0 1; do
for i in "${!macs[@]}"; do
[ "${cur[i]}" = "${want[i]}" ] && continue
link_rename "${cur[i]}" "${want[i]}" || return 1
done
for i in 0 1; do
for i in "${!macs[@]}"; do
if [ "$(iface_of "${macs[i]}")" != "${want[i]}" ]; then
log "ERROR: MAC ${macs[i]} is not named ${want[i]} after renaming"
return 1
@@ -216,8 +229,49 @@ set_names() {
}
names_ok() {
[ "$(iface_of "$PXE_MAC")" = "$PXE_NAME" ] &&
[ "$(iface_of "$PXE_MAC")" = "$PXE_NAME" ] || return 1
if [ -n "$CORP_MAC" ]; then
[ "$(iface_of "$CORP_MAC")" = "$CORP_NAME" ]
else
[ -z "$(stray_names)" ]
fi
}
# No pinned GE NIC: "MAC NAME" for every other physical NIC whose name the
# catch-all (en*, eth*) would not match - the previous imaging NIC still
# called pxe0 after a re-pick, or corp0 from a pinned config. Each gets the
# kernel-style MAC name enx<mac>, so it becomes a catch-all DHCP client and
# pxe0 is free for the new imaging NIC.
stray_names() {
local d n m
[ -z "$CORP_MAC" ] || return 0
for d in "$SYSFS_NET"/*; do
[ -e "$d/device" ] || continue
n=${d##*/}
m=$(cat "$d/address" 2>/dev/null)
[ "$m" = "$PXE_MAC" ] && continue
[ "$(cat "$d/type" 2>/dev/null)" = 1 ] || continue
[ -e "$d/wireless" ] || [ -e "$d/phy80211" ] || [ -e "$d/bridge" ] && continue
case "$n" in en*|eth*) continue ;; esac
echo "$m enx${m//:/}"
done
return 0
}
# MACs of the physical NICs other than the imaging one that hold a DHCP
# (dynamic) IPv4: the GE side. They must have one again after the apply,
# or an admin on the GE side may have been cut off. Static addresses (the
# previous imaging NIC's 172.16.9.1) do not count. By MAC: names change.
other_v4_macs() {
local d n m
for d in "$SYSFS_NET"/*; do
[ -e "$d/device" ] || continue
n=${d##*/}
m=$(cat "$d/address" 2>/dev/null)
[ "$m" = "$PXE_MAC" ] && continue
ip -4 -o addr show dev "$n" scope global dynamic 2>/dev/null | grep -q ' inet ' && echo "$m"
done
return 0
}
# --- backup / restore --------------------------------------------------------
@@ -239,6 +293,7 @@ write_state() {
echo "ORIG_PXE_IF=$ORIG_PXE_IF"
echo "CORP_MAC=$CORP_MAC"
echo "ORIG_CORP_IF=$ORIG_CORP_IF"
echo "EXTRA=$EXTRA"
} >"$BACKUP.state"
}
@@ -249,6 +304,7 @@ read_state() {
NETPLAN_DIR) [[ "$v" =~ ^/[A-Za-z0-9/._-]*$ ]] && NETPLAN_DIR=$v ;;
PXE_MAC|CORP_MAC) [[ "$v" =~ ^([0-9a-f]{2}:){5}[0-9a-f]{2}$ ]] && printf -v "$k" '%s' "$v" ;;
ORIG_PXE_IF|ORIG_CORP_IF) [[ "$v" =~ ^[A-Za-z0-9._-]{0,15}$ ]] && printf -v "$k" '%s' "$v" ;;
EXTRA) [[ "$v" =~ ^(([0-9a-f]{2}:){5}[0-9a-f]{2}=[A-Za-z0-9._-]{1,15},?)*$ ]] && EXTRA=$v ;;
esac
done <"$BACKUP.state"
TARGET=$NETPLAN_DIR/50-cloud-init.yaml
@@ -268,8 +324,11 @@ restore() {
log "restoring the previous config from $BACKUP"
find "$NETPLAN_DIR" -maxdepth 1 -type f \( -name '*.yaml' -o -name '*.yml' \) -delete
cp -a "$BACKUP/." "$NETPLAN_DIR/"
if [ -n "${ORIG_PXE_IF:-}" ] && [ -n "${ORIG_CORP_IF:-}" ]; then
set_names "$PXE_MAC" "$ORIG_PXE_IF" "$CORP_MAC" "$ORIG_CORP_IF" ||
if [ -n "${ORIG_PXE_IF:-}" ]; then
local -a back=()
local pair
for pair in ${EXTRA//,/ }; do back+=("${pair%%=*}" "${pair#*=}"); done
set_names "$PXE_MAC" "$ORIG_PXE_IF" "$CORP_MAC" "${ORIG_CORP_IF:-}" "${back[@]}" ||
{ log "ERROR: could not put the previous NIC names back"; rc=1; }
fi
netplan generate && netplan apply || { log "ERROR: netplan apply of the restored config failed"; rc=1; }
@@ -300,7 +359,7 @@ case "${1:-}" in
log "revert: $BACKUP already committed or reverted; nothing to do"
exit 0
fi
PXE_MAC="" CORP_MAC="" ORIG_PXE_IF="" ORIG_CORP_IF=""
PXE_MAC="" CORP_MAC="" ORIG_PXE_IF="" ORIG_CORP_IF="" EXTRA=""
read_state
log "revert: the apply was not confirmed within its window"
restore
@@ -308,15 +367,16 @@ case "${1:-}" in
;;
esac
if [ $# -ne 5 ]; then
echo "usage: $0 [--preflight] STAGED_YAML PXE_NAME PXE_MAC CORP_NAME CORP_MAC" >&2
if [ $# -ne 3 ] && [ $# -ne 5 ]; then
echo "usage: $0 [--preflight] STAGED_YAML PXE_NAME PXE_MAC [CORP_NAME CORP_MAC]" >&2
exit 2
fi
STAGED=$1
PXE_NAME=$2
PXE_MAC=${3,,}
CORP_NAME=$4
CORP_MAC=${5,,}
CORP_NAME=${4:-}
CORP_MAC=${5:-}
CORP_MAC=${CORP_MAC,,}
# 1. validate in a scratch root
SCRATCH=$(mktemp -d)
@@ -330,7 +390,7 @@ fi
# 2. already in place?
if cmp -s "$STAGED" "$TARGET" && [ -z "$(other_files)" ] && names_ok; then
log "netplan already in place, $PXE_NAME and $CORP_NAME present"
log "netplan already in place, $PXE_NAME${CORP_NAME:+ and $CORP_NAME} present"
echo UNCHANGED
exit 0
fi
@@ -350,14 +410,26 @@ flock -w 60 9 || { log "ERROR: another pxe-netplan-apply holds $LOCK_FILE; nothi
# 4. back up, record names, arm the auto-revert
ORIG_PXE_IF=$(iface_of "$PXE_MAC")
ORIG_CORP_IF=$(iface_of "$CORP_MAC")
ORIG_CORP_IF=""
[ -n "$CORP_MAC" ] && ORIG_CORP_IF=$(iface_of "$CORP_MAC")
CORP_HAD_V4=0
has_v4 "$ORIG_CORP_IF" && CORP_HAD_V4=1
[ -n "$ORIG_CORP_IF" ] && has_v4 "$ORIG_CORP_IF" && CORP_HAD_V4=1
OTHERS_V4=""
[ -z "$CORP_MAC" ] && OTHERS_V4=$(other_v4_macs)
# NICs to move off a name the catch-all would not match, recorded with
# their current names for the revert ("mac=name,...").
EXTRA=""
FREE=()
while read -r m newname; do
[ -n "$m" ] || continue
EXTRA="${EXTRA:+$EXTRA,}$m=$(iface_of "$m")"
FREE+=("$m" "$newname")
done < <(stray_names)
BACKUP=$BACKUP_ROOT/netplan-$(date +%Y%m%d-%H%M%S)
mkdir -p "$BACKUP"
cp -a "$NETPLAN_DIR/." "$BACKUP/"
write_state
log "previous config saved in $BACKUP (NICs: $ORIG_PXE_IF, $ORIG_CORP_IF)"
log "previous config saved in $BACKUP (NICs: $ORIG_PXE_IF${ORIG_CORP_IF:+, $ORIG_CORP_IF})"
REVERT_UNIT=""
if [ "$REVERT_AFTER" -gt 0 ] 2>/dev/null && command -v systemd-run >/dev/null; then
@@ -389,20 +461,32 @@ netplan generate || fail "netplan generate failed"
# 6. stable names now, so dnsmasq/UFW/Samba (which use them) work before
# the next reboot. Does not depend on the netplan version renaming.
set_names "$PXE_MAC" "$PXE_NAME" "$CORP_MAC" "$CORP_NAME" || fail "NIC rename failed"
[ ${#FREE[@]} -gt 0 ] && log "freeing names for the catch-all: $EXTRA"
set_names "$PXE_MAC" "$PXE_NAME" "$CORP_MAC" "$CORP_NAME" "${FREE[@]}" || fail "NIC rename failed"
# 7. apply
netplan apply || fail "netplan apply failed"
# 8. check
names_ok || fail "$PXE_NAME / $CORP_NAME are not on MACs $PXE_MAC / $CORP_MAC after apply"
names_ok || fail "$PXE_NAME${CORP_NAME:+ / $CORP_NAME} not on MAC $PXE_MAC${CORP_MAC:+ / $CORP_MAC} after apply"
ok=0
for _ in $(seq 1 "$PXE_WAIT"); do
if has_v4 "$PXE_NAME"; then ok=1; break; fi
sleep 1
done
[ "$ok" -eq 1 ] || fail "$PXE_NAME has no IPv4 address after ${PXE_WAIT}s"
if [ "$CORP_HAD_V4" -eq 1 ]; then
for m in $OTHERS_V4; do
ok=0
for _ in $(seq 1 "$CORP_WAIT"); do
n=$(iface_of "$m")
if [ -n "$n" ] && has_v4 "$n"; then ok=1; break; fi
sleep 1
done
[ "$ok" -eq 1 ] || fail "the NIC with MAC $m had a DHCP IPv4 before and has none after ${CORP_WAIT}s"
done
if [ -z "$CORP_MAC" ]; then
:
elif [ "$CORP_HAD_V4" -eq 1 ]; then
ok=0
for _ in $(seq 1 "$CORP_WAIT"); do
if has_v4 "$CORP_NAME"; then ok=1; break; fi
@@ -427,6 +511,6 @@ touch "$BACKUP.committed"
cancel_revert
trap - INT TERM
ip -br addr show dev "$PXE_NAME" 2>/dev/null
ip -br addr show dev "$CORP_NAME" 2>/dev/null
[ -n "$CORP_NAME" ] && ip -br addr show dev "$CORP_NAME" 2>/dev/null
echo CHANGED
exit 0

297
playbook/pxe-server-helpers/pxe-pick-nic Executable file
View File

@@ -0,0 +1,297 @@
#!/bin/bash
# pxe-pick-nic - choose the imaging NIC by its cable, not by its MAC.
#
# Usage:
# sudo pxe-pick-nic interactive: pull the imaging switch cable and
# plug it back in; the port that blinked is it
# pxe-pick-nic --auto for the first-boot playbook: the one wired
# port with a link, if it is isolated
# pxe-pick-nic --show print the saved choice
#
# The choice is saved in /etc/pxe-server/nic-pick.conf (imaging_mac=...).
# The playbook uses it when site-vars.yml leaves pxe_nic_mac empty, and pins
# that MAC from then on: the box never re-guesses at boot, so a moved cable
# can never move 172.16.9.1 and DHCP onto another port.
#
# A port is accepted only when pxe-nic-check calls it "isolated": no GE
# switch (LLDP/CDP), no 802.1X port authentication, and no other DHCP
# server answering one DHCPDISCOVER. It listens CHECK_WINDOW seconds
# (default 65: a Cisco switch sends CDP every 60 s). --auto also refuses a
# port where more than ARP_MAX (default 10) hosts were heard: an imaging
# switch at install time is nearly silent, a GE access VLAN is not, and
# that holds even when the switch sends no LLDP/CDP and the probe is lost
# (STP still listening). Anything else is refused and nothing is saved.
# The check uses the site's corp_domain_suffixes and allowed_switches
# (PXE_PICK_SUFFIXES / PXE_PICK_ALLOWED from the playbook, else
# /etc/pxe-server/nic.conf), so a managed imaging switch can be allowed.
#
# Install rule for sites: during the install plug in ONLY the imaging
# switch. Plug the GE cable into any other port afterwards; the catch-all
# netplan entry makes it a DHCP client and dnsmasq never serves it.
#
# Exit: 0 chosen (or --show found one), 1 refused / could not choose,
# 2 usage.
set -u
CONF=${PXE_PICK_CONF:-/etc/pxe-server/nic-pick.conf}
HINT=${PXE_PICK_HINT:-/var/lib/pxe-firstboot/nic-hint}
SYSFS_NET=${SYSFS_NET:-/sys/class/net}
NIC_CHECK=${PXE_NIC_CHECK:-/usr/local/sbin/pxe-nic-check}
CHECK_WINDOW=${CHECK_WINDOW:-65}
ARP_MAX=${PXE_PICK_ARP_MAX:-10}
NIC_CONF=${PXE_NIC_CONF:-/etc/pxe-server/nic.conf}
SETTLE=${PXE_PICK_SETTLE:-8}
WAIT=${PXE_PICK_WAIT:-180}
log() { echo "pxe-pick-nic: $*"; }
# Physical wired Ethernet ports: a device behind them, ARP type 1, not
# wireless, not a bridge.
wired() {
local d n
for d in "$SYSFS_NET"/*; do
n=${d##*/}
[ -e "$d/device" ] || continue
[ "$(cat "$d/type" 2>/dev/null)" = 1 ] || continue
[ -e "$d/wireless" ] || [ -e "$d/phy80211" ] && continue
[ -e "$d/bridge" ] && continue
echo "$n"
done
}
mac_of() { cat "$SYSFS_NET/$1/address" 2>/dev/null; }
# 1 = link, 0 = none. carrier is unreadable while the port is admin-down.
carrier() { [ "$(cat "$SYSFS_NET/$1/carrier" 2>/dev/null)" = 1 ] && echo 1 || echo 0; }
# Bring every wired port up (no address is added) so its link can be seen.
links_up() {
local n up=0
for n in $(wired); do
if ! ip link show dev "$n" 2>/dev/null | grep -q '[<,]UP[,>]'; then
ip link set dev "$n" up 2>/dev/null && up=1
fi
done
[ "$up" -eq 1 ] && sleep "$SETTLE"
return 0
}
table() {
local n
for n in $(wired); do
printf ' %-16s %s link %s\n' "$n" "$(mac_of "$n")" \
"$([ "$(carrier "$n")" = 1 ] && echo up || echo down)"
done
}
# A pxe-nic-check config with only the site settings that matter here
# (never the MAC keys: the port under test is given with --iface).
check_config() {
local f=$1 suffixes=${PXE_PICK_SUFFIXES:-} allowed=${PXE_PICK_ALLOWED:-}
if [ -z "$suffixes$allowed" ] && [ -f "$NIC_CONF" ]; then
suffixes=$(sed -n 's/^[[:space:]]*corp_domain_suffixes[[:space:]]*=[[:space:]]*//p' "$NIC_CONF" | tail -1)
allowed=$(sed -n 's/^[[:space:]]*allowed_switches[[:space:]]*=[[:space:]]*//p' "$NIC_CONF" | tail -1)
fi
{
echo "[nic]"
[ -n "$suffixes" ] && echo "corp_domain_suffixes = $suffixes"
[ -n "$allowed" ] && echo "allowed_switches = $allowed"
} >"$f"
}
# check IFACE [strict]: prints the verdict, returns 0 only for "isolated"
# (with strict, also only when at most ARP_MAX hosts were heard).
check() {
local out verdict arp why cfg
if [ ! -f "$NIC_CHECK" ]; then
log "ERROR: $NIC_CHECK not found"
return 1
fi
cfg=$(mktemp)
check_config "$cfg"
log "checking $1 is isolated: listening ${CHECK_WINDOW}s for GE switches and DHCP servers..."
out=$(python3 "$NIC_CHECK" --iface "$1" --probe --window "$CHECK_WINDOW" --json --config "$cfg" 2>/dev/null)
rm -f "$cfg"
read -r verdict arp why < <(python3 -c '
import json, sys
try:
e = json.load(sys.stdin)["interfaces"][0]
ev = "; ".join(x for x in e["evidence"] if not x.endswith("ARP hosts seen"))
print(e["verdict"], int(e.get("arp_hosts") or 0), ev or "-")
except Exception as ex:
print("error", 0, "pxe-nic-check gave no usable result: %s" % ex)
' <<<"$out")
if [ "$verdict" = isolated ]; then
if [ "$arp" -gt "$ARP_MAX" ]; then
log "$1: isolated by its switch and DHCP, but $arp hosts were heard (more than $ARP_MAX)"
if [ "${2:-}" = strict ]; then
log "REFUSED: a busy LAN is not an imaging switch at install time; it may be a GE port"
log "with LLDP/CDP off. Pick it by its cable (sudo pxe-pick-nic) or set pxe_nic_mac."
return 1
fi
log "WARNING: make sure this is the imaging switch, not a GE port with LLDP/CDP turned off."
else
log "$1: isolated (no GE switch, no 802.1X, no other DHCP server, $arp hosts heard)"
fi
return 0
fi
log "$1: verdict $verdict: $why"
case "$verdict" in
ge) log "That port is on the GE network (or a managed switch not in allowed_switches). It must never be the imaging port." ;;
foreign-dhcp) log "Another DHCP server answers on that port (a second PXE server, or a routed LAN)." ;;
no-link) log "That port lost its link during the check." ;;
esac
return 1
}
save() {
local n=$1 mac
mac=$(mac_of "$n")
mkdir -p "$(dirname "$CONF")"
{
echo "# Written by pxe-pick-nic $(date -Is). The imaging NIC; the playbook"
echo "# pins it when site-vars.yml leaves pxe_nic_mac empty."
echo "imaging_mac=$mac"
echo "picked_iface=$n"
} >"$CONF.tmp" && mv "$CONF.tmp" "$CONF"
rm -f "$HINT"
log "saved: imaging NIC = $n ($mac) in $CONF"
}
hint() {
mkdir -p "$(dirname "$HINT")" 2>/dev/null
{ printf '%s\n' "$*" >"$HINT"; } 2>/dev/null
}
saved_mac() {
sed -n 's/^imaging_mac=\(\([0-9a-f]\{2\}:\)\{5\}[0-9a-f]\{2\}\)$/\1/p' "$CONF" 2>/dev/null | tail -1
}
# --- modes ---------------------------------------------------------------------
mode_show() {
local mac
mac=$(saved_mac)
[ -n "$mac" ] || { log "no saved choice in $CONF"; return 1; }
echo "$mac"
}
mode_auto() {
local mac n with=() names
mac=$(saved_mac)
if [ -n "$mac" ]; then
for n in $(wired); do
if [ "$(mac_of "$n")" = "$mac" ]; then
log "using the saved choice: $n ($mac)"
echo "$mac"
return 0
fi
done
log "the saved imaging NIC $mac is not on this box any more"
hint "The saved imaging NIC $mac is gone (dongle replaced?). Log in and run: sudo pxe-pick-nic"
return 1
fi
links_up
for n in $(wired); do
[ "$(carrier "$n")" = 1 ] && with+=("$n")
done
log "wired ports:"
table
if [ ${#with[@]} -eq 0 ]; then
log "REFUSED: no wired port has a link. Plug the imaging switch into the port to use."
hint "Could not pick the imaging port: no port has a link. Plug in the imaging switch, then log in and run: sudo pxe-pick-nic"
return 1
fi
if [ ${#with[@]} -gt 1 ]; then
names="${with[*]}"
log "REFUSED: ${#with[@]} ports have a link ($names). During the install only the imaging switch may be plugged in."
hint "Could not pick the imaging port: ${#with[@]} ports have a link ($names). Log in and run: sudo pxe-pick-nic"
return 1
fi
n=${with[0]}
if ! check "$n" strict; then
hint "Could not pick the imaging port: $n is not isolated (see /var/log/first-boot.log). Plug only the imaging switch in, then log in and run: sudo pxe-pick-nic"
return 1
fi
save "$n" >&2
mac_of "$n"
}
# Wait until exactly one port changes its link; print "NAME NEWSTATE".
wait_change() {
local -A before=()
local n t=0 changed=()
for n in $(wired); do before[$n]=$(carrier "$n"); done
while [ "$t" -lt $((WAIT * 2)) ]; do
sleep 0.5
t=$((t + 1))
changed=()
for n in "${!before[@]}"; do
[ "$(carrier "$n")" != "${before[$n]}" ] && changed+=("$n")
done
if [ ${#changed[@]} -eq 1 ]; then
sleep 1 # a flapping connector settles
n=${changed[0]}
echo "$n $(carrier "$n")"
return 0
fi
if [ ${#changed[@]} -gt 1 ]; then
log "more than one port changed (${changed[*]}); move one cable at a time" >&2
return 1
fi
done
log "nothing changed in ${WAIT}s" >&2
return 1
}
mode_interactive() {
local n state again ans
[ -t 0 ] || { log "interactive mode needs a terminal (use --auto from scripts)"; return 2; }
[ "$(id -u)" -eq 0 ] || { log "run it as root: sudo pxe-pick-nic"; return 2; }
links_up
echo
echo "Pick the imaging NIC: the port cabled to the isolated imaging switch."
echo "Wired ports now:"
table
echo
echo "Pull the imaging switch cable out of this server now"
echo "(or, if it is not plugged in yet, plug it in now). Waiting up to ${WAIT}s..."
read -r n state < <(wait_change) || return 1
[ -n "${n:-}" ] || return 1
if [ "$state" = 0 ]; then
echo "$n lost its link. Plug the cable back into the SAME port..."
read -r again state < <(wait_change) || return 1
if [ "${again:-}" != "$n" ] || [ "$state" != 1 ]; then
log "expected $n to come back up, saw ${again:-nothing} $state; try again"
return 1
fi
fi
sleep "$SETTLE"
echo "Imaging port: $n ($(mac_of "$n"))"
check "$n" || { log "nothing saved"; return 1; }
read -rp "Use $n ($(mac_of "$n")) as the imaging NIC? [y/N] " ans
[[ "$ans" =~ ^[Yy]$ ]] || { log "nothing saved"; return 1; }
save "$n"
if [ -e /var/lib/pxe-firstboot.failed ] && [ -x /opt/first-boot.sh ]; then
read -rp "Run the server setup again now? [Y/n] " ans
if [[ ! "$ans" =~ ^[Nn]$ ]]; then
rm -f /var/lib/pxe-firstboot/attempts
/opt/first-boot.sh 2>&1 | tee -a /var/log/first-boot.log
return "${PIPESTATUS[0]}"
fi
echo "Run it later with: sudo rm -f /var/lib/pxe-firstboot/attempts; sudo /opt/first-boot.sh"
else
echo "Saved. If the server is already set up, re-run the playbook from the"
echo "installer USB to move the imaging config onto this port."
fi
return 0
}
case "${1:-}" in
--auto) mode_auto ;;
--show) mode_show ;;
"") mode_interactive ;;
-h|--help) sed -n '2,/^$/p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) echo "usage: $0 [--auto | --show]" >&2; exit 2 ;;
esac

246
playbook/pxe_server_setup.yml
View File

@@ -73,19 +73,30 @@
# image roots (on a rebuild the _media views may not exist yet).
zscaler_ca_share_glob: "{{ samba_share }}/_media/*/Deploy/Applications/ZscalerCommercialCertificate-2048-SHA256.crt {{ samba_share }}/gea-*/Deploy/Applications/ZscalerCommercialCertificate-2048-SHA256.crt"
# --- The two NICs (site standard) ---
# Every server has two NICs: the imaging NIC (isolated imaging LAN,
# static pxe_server_ip, the ONLY NIC that serves DHCP/TFTP/HTTP boot/SMB)
# and the GE corporate NIC (DHCP client; SSH and the webapp on 9009 for
# admins, outbound updates through Zscaler). Both MACs are REQUIRED:
# set them in playbook/site-vars.yml on the USB (copy
# site-vars.example.yml) or with -e. The playbook stops before it
# touches anything if either is unset or not on this box. Read them
# with: ip -br link
# The imaging NIC (isolated imaging LAN, static pxe_server_ip, the ONLY
# NIC that serves DHCP/TFTP/HTTP boot/SMB) is pinned by MAC. Leave
# pxe_nic_mac empty and the first run picks it (pxe-pick-nic --auto):
# the one wired port with a link, accepted only if pxe-nic-check calls
# it isolated, saved in /etc/pxe-server/nic-pick.conf and pinned from
# then on. So during the install plug in ONLY the imaging switch. If it
# cannot tell, it stops and the console says to run sudo pxe-pick-nic
# (pull the imaging cable and plug it back in).
#
# The GE corporate NIC (DHCP client; SSH and the webapp on 9009) is
# optional:
# corp_nic_mac empty (default): every other wired port (en*, eth*) is
# a DHCP client, so GE can be plugged into any of them, then or
# later. dnsmasq never serves them (interface= pins it to the
# imaging NIC) and they ignore our own DHCP offers.
# corp_nic_mac set: that one port is pinned and named corp_nic_name,
# as on the WJ server; other ports stay unconfigured.
pxe_nic_mac: ""
corp_nic_mac: ""
# Stable names, set by netplan (match macaddress + set-name). dnsmasq,
# UFW and Samba refer to these. A site may keep its existing names
# (e.g. pxe_nic_name: enx34c8d6b11010, corp_nic_name: enp0s31f6).
# UFW and Samba refer to these. With corp_nic_mac empty they must not
# look like en*/eth* (the catch-all would match them by name). With both
# MACs pinned a site may keep its existing names (WJ: pxe_nic_name:
# enx34c8d6b11010, corp_nic_name: enp0s31f6).
pxe_nic_name: "pxe0"
corp_nic_name: "corp0"
# Imaging LAN addressing. NOTE: 172.16.9.1 is also hard-coded in the
@@ -101,6 +112,7 @@
corp_route_metric: 100
# NIC guard (pxe-nic-check): dnsmasq must never serve DHCP/PXE on the GE
# corporate network. These are derived from pxe_nic_mac / corp_nic_mac
# (pxe_ge_mac empty: the guard takes the NIC holding a DHCP lease)
# (one source of truth) and written to /etc/pxe-server/nic.conf; do not
# set them separately (the playbook fails if they disagree). A MAC of a
# bridge port resolves to the bridge.
@@ -210,29 +222,77 @@
- Tools
tasks:
# --- The two NICs: validate, then pin by MAC (netplan) ---
# --- The NICs: pick / validate, then pin by MAC (netplan) ---
# Replaces the old guess (first en* without 172.16.9.1 / the default
# route), which on a two-NIC box could pick the GE NIC.
# route), which on a two-NIC box could pick the GE NIC. The pick runs
# from the USB copies, before anything on the box changes, and only
# ever at install time: once pinned, nothing re-guesses at boot.
# Installed first (they change no network config): when the pick
# refuses, the play stops here and the console tells the operator to
# run sudo pxe-pick-nic, which needs both on the box (first-boot.sh
# unmounts the USB).
- name: "NICs: install pxe-nic-check and pxe-pick-nic before the pick"
copy:
src: "{{ usb_mount }}/pxe-server-helpers/{{ item }}"
dest: "/usr/local/sbin/{{ item }}"
mode: '0755'
loop: [pxe-nic-check, pxe-pick-nic]
- name: "NICs: pick the imaging NIC (pxe_nic_mac is not set in site-vars)"
command: /usr/local/sbin/pxe-pick-nic --auto
environment:
PXE_PICK_SUFFIXES: "{{ pxe_corp_domain_suffixes }}"
PXE_PICK_ALLOWED: "{{ pxe_nic_allowed_switches }}"
register: nic_pick
changed_when: false
failed_when: false
when: pxe_nic_mac | length == 0
- name: "NICs: pick result"
debug:
msg: "{{ nic_pick.stdout_lines }}"
when: pxe_nic_mac | length == 0
- name: "NICs: the imaging NIC could not be picked"
fail:
msg: >-
Could not pick the imaging NIC, nothing was changed:
{{ nic_pick.stdout_lines | select('search', 'REFUSED|verdict|gone|ERROR') | join(' ') }}
During the install plug in ONLY the imaging switch, or log in and
run: sudo pxe-pick-nic (pull the imaging cable and plug it back in).
Or set pxe_nic_mac in playbook/site-vars.yml.
when:
- pxe_nic_mac | length == 0
- nic_pick.rc != 0 or not (nic_pick.stdout_lines | last | default('') is match('^([0-9a-f]{2}:){5}[0-9a-f]{2}$'))
- name: "NICs: pin the picked imaging NIC"
set_fact:
pxe_nic_mac: "{{ nic_pick.stdout_lines | last }}"
when: pxe_nic_mac | length == 0
- name: "NICs: check the per-site NIC vars"
assert:
that:
- pxe_nic_mac | length > 0
- corp_nic_mac | length > 0
- pxe_nic_mac is match('^([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}$')
- corp_nic_mac is match('^([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}$')
- corp_nic_mac | length == 0 or corp_nic_mac is match('^([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}$')
- pxe_nic_mac | lower != corp_nic_mac | lower
- pxe_imaging_mac | lower == pxe_nic_mac | lower
- pxe_ge_mac | lower == corp_nic_mac | lower
- pxe_nic_name | length > 0 and pxe_nic_name | length <= 15
- corp_nic_name | length > 0 and corp_nic_name | length <= 15
- pxe_nic_name != corp_nic_name
- corp_nic_mac | length > 0 or not (pxe_nic_name is match('^(en|eth)'))
fail_msg: >-
Set pxe_nic_mac (imaging NIC) and corp_nic_mac (GE corporate NIC)
to two different MACs (aa:bb:cc:dd:ee:ff) in playbook/site-vars.yml
(copy site-vars.example.yml) or with -e, then re-run. Do not set
pxe_imaging_mac / pxe_ge_mac yourself; they follow these. Names
(pxe_nic_name, corp_nic_name) must differ and be 1-15 chars.
Now: pxe_nic_mac='{{ pxe_nic_mac }}' corp_nic_mac='{{ corp_nic_mac }}'.
pxe_nic_mac (imaging NIC) must be a MAC (aa:bb:cc:dd:ee:ff);
corp_nic_mac (GE NIC) empty or a different MAC. Set them in
playbook/site-vars.yml (copy site-vars.example.yml) or with -e,
then re-run. Do not set pxe_imaging_mac / pxe_ge_mac yourself;
they follow these (an -e pxe_nic_mac="" overrides the pick too:
drop it). Names (pxe_nic_name, corp_nic_name) must differ
and be 1-15 chars, and with corp_nic_mac empty pxe_nic_name must
not start with en or eth (the catch-all for the other ports
matches those). Now: pxe_nic_mac='{{ pxe_nic_mac }}'
corp_nic_mac='{{ corp_nic_mac }}' pxe_nic_name='{{ pxe_nic_name }}'.
quiet: yes
- name: "NICs: list the physical NICs on this box"
@@ -244,15 +304,15 @@
register: nic_list
changed_when: false
- name: "NICs: both MACs must be on this box"
- name: "NICs: the pinned MACs must be on this box"
vars:
box_macs: "{{ nic_list.stdout_lines | map('regex_replace', '^.* ', '') | map('lower') | list }}"
assert:
that:
- pxe_nic_mac | lower in box_macs
- corp_nic_mac | lower in box_macs
- corp_nic_mac | length == 0 or corp_nic_mac | lower in box_macs
fail_msg: >-
{{ 'pxe_nic_mac ' ~ pxe_nic_mac ~ ' is not on this box. ' if pxe_nic_mac | lower not in box_macs else '' }}{{ 'corp_nic_mac ' ~ corp_nic_mac ~ ' is not on this box. ' if corp_nic_mac | lower not in box_macs else '' }}Nothing was changed (the network config would have left that NIC
{{ 'pxe_nic_mac ' ~ pxe_nic_mac ~ ' is not on this box. ' if pxe_nic_mac | lower not in box_macs else '' }}{{ 'corp_nic_mac ' ~ corp_nic_mac ~ ' is not on this box. ' if corp_nic_mac and corp_nic_mac | lower not in box_macs else '' }}Nothing was changed (the network config would have left that NIC
unconfigured). Physical NICs here (name MAC):
{{ nic_list.stdout_lines | join('; ') }}.
Fix playbook/site-vars.yml and re-run.
@@ -396,7 +456,7 @@
bash {{ usb_mount }}/pxe-server-helpers/pxe-netplan-apply.sh --preflight
/run/pxe-server-netplan.check.yaml
{{ pxe_nic_name }} {{ pxe_nic_mac | lower }}
{{ corp_nic_name }} {{ corp_nic_mac | lower }}
{{ (corp_nic_name ~ ' ' ~ corp_nic_mac | lower) if corp_nic_mac else '' }}
environment:
PXE_SUBNET: "{{ pxe_subnet }}"
PXE_SSH_CLIENT: "{{ pxe_ssh_client }}"
@@ -439,11 +499,57 @@
dest: /etc/pxe-server-netplan.yaml
mode: '0600'
# The catch-all ports (corp_nic_mac empty) are DHCP clients on whatever
# they are cabled to. Cabled to the imaging switch by mistake, they would
# take an address from our own dnsmasq and put a second interface in the
# imaging subnet; DenyList makes them ignore offers from pxe_server_ip.
- name: "NICs: drop-in folders for the catch-all ports"
file:
path: "/etc/systemd/network/10-netplan-{{ item }}.network.d"
state: directory
mode: '0755'
loop: [zz-en, zz-eth]
when: corp_nic_mac | length == 0
- name: "NICs: catch-all ports ignore our own DHCP offers"
copy:
dest: "/etc/systemd/network/10-netplan-{{ item }}.network.d/90-pxe-deny-own-dhcp.conf"
mode: '0644'
content: |
# Managed by pxe_server_setup.yml. Never take a lease from this
# server's own dnsmasq (an imaging switch on a non-imaging port).
[DHCPv4]
DenyList={{ pxe_server_ip }}
loop: [zz-en, zz-eth]
when: corp_nic_mac | length == 0
- name: "NICs: no catch-all drop-ins with a pinned GE NIC"
file:
path: "/etc/systemd/network/10-netplan-{{ item }}.network.d"
state: absent
loop: [zz-en, zz-eth]
when: corp_nic_mac | length > 0
- name: "NICs: create /etc/pxe-server"
file:
path: /etc/pxe-server
state: directory
mode: '0755'
- name: "NICs: record the pinned imaging NIC for pxe-pick-nic --show"
copy:
dest: /etc/pxe-server/nic-pick.conf
mode: '0644'
content: |
# Written by pxe_server_setup.yml. The imaging NIC the playbook pins.
imaging_mac={{ pxe_nic_mac | lower }}
when: nic_pick is not skipped and nic_pick.rc == 0
- name: "NICs: validate, install and apply netplan (rolls back on failure)"
command: >-
/usr/local/sbin/pxe-netplan-apply.sh /etc/pxe-server-netplan.yaml
{{ pxe_nic_name }} {{ pxe_nic_mac | lower }}
{{ corp_nic_name }} {{ corp_nic_mac | lower }}
{{ (corp_nic_name ~ ' ' ~ corp_nic_mac | lower) if corp_nic_mac else '' }}
environment:
PXE_SUBNET: "{{ pxe_subnet }}"
PXE_SSH_CLIENT: "{{ pxe_ssh_client }}"
@@ -520,7 +626,7 @@
listen-address={{ pxe_server_ip }}
bind-dynamic
# Interface pinning (serve {{ pxe_nic_name }} only, never
# {{ corp_nic_name }}) is in /etc/dnsmasq.d/pxe-nic-only.conf.
# {{ corp_nic_name if corp_nic_mac else 'the other ports' }}) is in /etc/dnsmasq.d/pxe-nic-only.conf.
dhcp-range={{ pxe_dhcp_start }},{{ pxe_dhcp_end }},{{ pxe_dhcp_lease }}
# No default gateway (option 3) and no DNS (option 6) handed out:
# the PXE network is isolated and the PXE server does not forward
@@ -576,6 +682,37 @@
mode: '0644'
backup: yes
# The second layer under interface=: dnsmasq offers only on an interface
# with an address inside a dhcp-range. With every range in the imaging
# subnet, a GE port (10.x) matches none even if the pin were lost. A
# proxy range or one outside the subnet would remove that layer.
- name: "dnsmasq: every dhcp-range is in the imaging subnet, none is proxy"
shell: |
set -u
bad=0
# Files the dnsmasq conf-dir skips (dot files, ~ backups, .dpkg-*,
# .bak) are skipped here too, so a backup never refuses the run.
FILES=(/etc/dnsmasq.conf)
for f in /etc/dnsmasq.d/*; do
b=${f##*/}
case "$b" in .*|*~|*.dpkg-*|*.bak|*.bak.*) continue ;; esac
[ -f "$f" ] && FILES+=("$f")
done
while IFS= read -r line; do
range=${line#*dhcp-range=}
first=$(printf '%s' "$range" | tr ',' '\n' | grep -Em1 '^[0-9]+(\.[0-9]+){3}$')
if printf '%s' "$range" | grep -qi proxy; then
echo "REFUSED proxy DHCP: $line" >&2; bad=1
elif [ -z "$first" ] || ! python3 -c 'import ipaddress,sys; sys.exit(ipaddress.ip_address(sys.argv[1]) not in ipaddress.ip_network(sys.argv[2]))' "$first" "{{ pxe_subnet }}"; then
echo "REFUSED dhcp-range outside {{ pxe_subnet }}: $line" >&2; bad=1
fi
done < <(grep -Hs '^[[:space:]]*dhcp-range=' "${FILES[@]}" | sed 's/[[:space:]]*#.*//')
[ "$bad" -eq 0 ] || exit 1
echo "dhcp-range ok"
args:
executable: /bin/bash
changed_when: false
# --- NIC guard: keep dnsmasq off the GE corporate network ---
# pxe-nic-check classifies each NIC from what it hears (LLDP/CDP/802.1X
# from managed switches, foreign DHCP offers and their domain). The
@@ -1927,7 +2064,21 @@
loop: "{{ ufw_corp_rules }}"
loop_control:
label: "{{ corp_nic_name }} {{ item.port }}/{{ item.proto }}"
when: corp_admin_cidrs | length == 0
when: corp_admin_cidrs | length == 0 and corp_nic_mac | length > 0
# No pinned GE NIC (corp_nic_mac empty): the GE port has no fixed name,
# so SSH and the webapp are allowed by source, on any interface. The
# imaging NIC already allows both, so this opens nothing new there.
- name: "UFW: allow SSH and the webapp from all of GE (no pinned GE NIC)"
ufw:
rule: allow
direction: in
port: "{{ item.port }}"
proto: "{{ item.proto }}"
loop: "{{ ufw_corp_rules }}"
loop_control:
label: "any {{ item.port }}/{{ item.proto }}"
when: corp_admin_cidrs | length == 0 and corp_nic_mac | length == 0
- name: "UFW: WARNING, SSH and the root webapp are open to all of GE"
debug:
@@ -1935,7 +2086,7 @@
- "!!! WARNING: corp_admin_cidrs is empty. SSH (password auth, shared"
- "!!! 'pxe' password) and the webapp on 9009 (no login, runs as root)"
- "!!! accept connections from ANY host on the GE corporate network via"
- "!!! {{ corp_nic_name }}. Set corp_admin_cidrs in playbook/site-vars.yml"
- "!!! {{ corp_nic_name if corp_nic_mac else 'any GE port' }}. Set corp_admin_cidrs in playbook/site-vars.yml"
- "!!! to the admin subnets (docs/SITE-CUSTOMIZATION.md) and re-run."
when: corp_admin_cidrs | length == 0
@@ -1950,7 +2101,19 @@
loop: "{{ ufw_corp_rules | product(corp_admin_cidrs) | list }}"
loop_control:
label: "{{ corp_nic_name }} from {{ item.1 }} {{ item.0.port }}/{{ item.0.proto }}"
when: corp_admin_cidrs | length > 0
when: corp_admin_cidrs | length > 0 and corp_nic_mac | length > 0
- name: "UFW: allow SSH and the webapp from corp_admin_cidrs only (no pinned GE NIC)"
ufw:
rule: allow
direction: in
from_ip: "{{ item.1 }}"
port: "{{ item.0.port }}"
proto: "{{ item.0.proto }}"
loop: "{{ ufw_corp_rules | product(corp_admin_cidrs) | list }}"
loop_control:
label: "any from {{ item.1 }} {{ item.0.port }}/{{ item.0.proto }}"
when: corp_admin_cidrs | length > 0 and corp_nic_mac | length == 0
- name: "UFW: drop the any-source rules on the GE corporate NIC (restricted now)"
ufw:
@@ -1963,7 +2126,7 @@
loop: "{{ ufw_corp_rules }}"
loop_control:
label: "{{ corp_nic_name }} any {{ item.port }}/{{ item.proto }}"
when: corp_admin_cidrs | length > 0
when: corp_admin_cidrs | length > 0 and corp_nic_mac | length > 0
# Only drop the old any-interface rules (allow 22 among them) once both
# named NICs exist on the right MACs and the per-interface SSH rules are
@@ -1975,29 +2138,46 @@
shell: |
set -u
bad=0
for pair in "{{ pxe_nic_name }}={{ pxe_nic_mac | lower }}" "{{ corp_nic_name }}={{ corp_nic_mac | lower }}"; do
PAIRS=("{{ pxe_nic_name }}={{ pxe_nic_mac | lower }}")
NAMES=("{{ pxe_nic_name }}")
{% if corp_nic_mac %}
PAIRS+=("{{ corp_nic_name }}={{ corp_nic_mac | lower }}")
NAMES+=("{{ corp_nic_name }}")
{% endif %}
for pair in "${PAIRS[@]}"; do
n=${pair%%=*}; m=${pair#*=}
if [ "$(cat "/sys/class/net/$n/address" 2>/dev/null)" != "$m" ]; then
echo "ERROR: interface $n with MAC $m is not present" >&2; bad=1
fi
done
added=$(ufw show added 2>/dev/null)
for n in "{{ pxe_nic_name }}" "{{ corp_nic_name }}"; do
for n in "${NAMES[@]}"; do
if ! grep -E "allow in on $n( |$)" <<<"$added" | grep -qE 'port 22 proto tcp'; then
echo "ERROR: no UFW 'allow in on $n ... port 22 proto tcp' rule" >&2; bad=1
fi
done
{% if not corp_nic_mac %}
# No pinned GE NIC: GE-side SSH is an any-interface rule, by source.
if ! grep -E '^ufw allow' <<<"$added" | grep -v ' on ' | grep -qE '(port 22( |$)| 22/tcp( |$))'; then
echo "ERROR: no any-interface UFW rule for port 22 (GE side)" >&2; bad=1
fi
{% endif %}
[ "$bad" -eq 0 ] || { echo "the old any-interface rules are kept" >&2; exit 1; }
echo "both NICs present, per-interface SSH rules in place"
register: ufw_verify
changed_when: false
# With no pinned GE NIC and corp_admin_cidrs empty, "allow 22" and
# "allow 9009" on any interface ARE the GE-side rules: keep them.
- name: "UFW: remove the old any-interface rules"
ufw:
rule: allow
port: "{{ item.port }}"
proto: "{{ item.proto }}"
delete: yes
when: >-
corp_nic_mac | length > 0 or corp_admin_cidrs | length > 0
or item.port not in (ufw_corp_rules | map(attribute='port') | list)
loop:
- { port: "22", proto: tcp }
- { port: "67", proto: udp }

20
playbook/site-vars.example.yml
View File

@@ -4,16 +4,22 @@
# loads it on first boot; -e on the command line still overrides it.
# site-vars.yml is git-ignored: each site's MACs stay with that site.
#
# REQUIRED: the two NICs, by MAC (ip -br link on the box, or the label).
# imaging NIC = cable to the isolated imaging switch (serves DHCP/PXE/SMB)
# corp NIC = cable to the GE corporate network (SSH + webapp 9009 only)
# The playbook changes nothing on the network until both are set, differ,
# and are both present on the box.
# The NICs, by MAC (ip -br link on the box, or the label). Both optional:
# pxe_nic_mac imaging NIC = cable to the isolated imaging switch (serves
# DHCP/PXE/SMB). Empty: the first boot picks the one wired
# port with a link, if pxe-nic-check calls it isolated, and
# pins it. So during the install plug in ONLY the imaging
# switch. If it cannot tell, the console says to log in and
# run: sudo pxe-pick-nic (pull the imaging cable, plug it in).
# corp_nic_mac GE corporate NIC (SSH + webapp 9009). Empty: every other
# wired port is a DHCP client, so GE can be plugged into any
# of them, now or later. Set: only that port, named corp0.
pxe_nic_mac: "" # e.g. "34:c8:d6:b1:10:10"
corp_nic_mac: "" # e.g. "20:88:10:af:0d:e5"
# Optional: stable interface names (default pxe0 / corp0). Keep a box's
# existing names instead, e.g. on the WJ server:
# Optional: stable interface names (default pxe0 / corp0). With corp_nic_mac
# empty, pxe_nic_name must not start with en or eth. With both MACs set a
# box may keep its existing names, e.g. on the WJ server:
# pxe_nic_name: enx34c8d6b11010
# corp_nic_name: enp0s31f6

6
playbook/templates/dnsmasq-pxe-nic-only.conf.j2
View File

@@ -4,5 +4,11 @@
# port=0 / listen-address / bind-dynamic in /etc/dnsmasq.conf and the
# pxe-nic-check guard (docs/pxe-nic-guard.md).
interface={{ pxe_nic_name }}
{% if corp_nic_mac %}
except-interface={{ corp_nic_name }}
no-dhcp-interface={{ corp_nic_name }}
{% else %}
# No pinned GE NIC (corp_nic_mac empty): every other wired port is a DHCP
# client with no fixed name. interface= alone keeps dnsmasq off them, and
# its only dhcp-range is the imaging subnet, so a GE address matches none.
{% endif %}

32
playbook/templates/netplan-pxe.yaml.j2
View File

@@ -3,6 +3,7 @@
# replaced on the next playbook run; the previous file is kept under
# /var/backups/netplan-<timestamp>/.
#
{% if corp_nic_mac %}
# Two NICs, each matched by MAC and given a stable name:
# pxe-lan imaging LAN, static, no gateway, no DNS. The ONLY NIC that
# serves DHCP/TFTP/HTTP boot/SMB. ignore-carrier: the address
@@ -11,6 +12,24 @@
# Apache without {{ pxe_server_ip }} (networkd
# ConfigureWithoutCarrier= + IgnoreCarrierLoss=).
# ge-corp GE corporate network, DHCP client (SSH + webapp 9009, updates).
{% else %}
# The imaging NIC pinned by MAC, every other wired port a DHCP client:
# pxe-lan imaging LAN, matched by MAC and given a stable name. Static,
# no gateway, no DNS. The ONLY NIC that serves DHCP/TFTP/HTTP
# boot/SMB. ignore-carrier: the address is put on (and kept)
# without link, so a switch that is off or a cable pulled at
# boot does not leave dnsmasq, Samba and Apache without
# {{ pxe_server_ip }} (networkd ConfigureWithoutCarrier= +
# IgnoreCarrierLoss=).
# zz-en, zz-eth every other wired port: DHCP client, so the GE cable
# can go into any of them, at install or later. No set-name.
# networkd applies the first matching .network file in name
# order: 10-netplan-pxe-lan sorts before 10-netplan-zz-*, and
# {{ pxe_nic_name }} does not look like en*/eth* (the playbook
# checks), so the imaging NIC never falls into these. dnsmasq
# never serves them (interface= pins it to {{ pxe_nic_name }}),
# and a drop-in makes them ignore offers from {{ pxe_server_ip }}.
{% endif %}
network:
version: 2
renderer: networkd
@@ -25,6 +44,7 @@ network:
optional: true
addresses:
- {{ pxe_server_ip }}/{{ pxe_prefix }}
{% if corp_nic_mac %}
ge-corp:
match:
macaddress: "{{ corp_nic_mac | lower }}"
@@ -32,3 +52,15 @@ network:
dhcp4: true
dhcp4-overrides:
route-metric: {{ corp_route_metric }}
{% else %}
{% for id, glob in [('zz-en', 'en*'), ('zz-eth', 'eth*')] %}
{{ id }}:
match:
name: "{{ glob }}"
dhcp4: true
dhcp6: false
optional: true
dhcp4-overrides:
route-metric: {{ corp_route_metric }}
{% endfor %}
{% endif %}