NICs: pick the imaging port at install, GE can go into any other port

Sites no longer need either MAC in site-vars.

pxe_nic_mac empty: before anything changes, the playbook runs
pxe-pick-nic --auto. It takes the one wired port with a link, accepts it
only if pxe-nic-check calls it isolated (no LLDP/CDP/802.1X, no other
DHCP server, at most 10 hosts heard in 65 s), and pins it. The choice is
saved in /etc/pxe-server/nic-pick.conf and never re-guessed at boot.
When it cannot tell (no link, two links, not isolated) nothing changes,
and the console banner says why and to run sudo pxe-pick-nic: pull the
imaging cable and plug it back in, and the port that blinked is the
imaging one. Both helpers are installed before the pick, so that works
after a refused first boot. The site's corp domains and allowed
switches are passed to the check.

corp_nic_mac empty: every other wired port (en*, eth*) is a catch-all
DHCP client, so GE can be plugged in at install or later.
- dnsmasq stays pinned with interface=pxe0. A new check refuses any
  dhcp-range outside the imaging subnet, and proxy ranges.
- The catch-all ports ignore offers from 172.16.9.1.
- UFW allows SSH and 9009 by source on any interface, and keeps those
  rules when the old any-interface rules are removed.
- pxe_nic_name must not look like en*/eth* in this mode.

pxe-netplan-apply.sh takes 3 or 5 args. Without a pinned GE NIC it:
- renames stray names (the old pxe0 after a re-pick, a leftover corp0)
  to enx<mac>, so the catch-all takes them, and restores them on revert;
- checks that every NIC that had a DHCP address before has one after,
  tracked by MAC.

With both MACs set (the WJ server), netplan, the dnsmasq pin and UFW
render byte-for-byte as before.

Tested in the pxetest VMs:
- Fresh install with no MACs and the GE link down: the imaging port was
  picked and pinned as pxe0 with 172.16.9.1, and dnsmasq bound DHCP to
  pxe0 only. GE plugged in afterwards got a lease, SSH worked, and the
  guard reported pxe0 isolated.
- --auto refuses with two links.
- The interactive pick works by pulling and replugging the link.
- A 3-arg re-apply is UNCHANGED.

Unit tests (fake sysfs) cover the re-pick and pinned-to-catch-all renames.
This commit is contained in:
cproudlock
2026-10-16 15:56:47 -04:00
parent 9af1c6d89a
commit b11e678297
12 changed files with 774 additions and 108 deletions

View File

@@ -34,12 +34,12 @@
# Engineering Layer...: scripts/capture-site-content.sh).
# Without it a new server cannot build standard or
# engineer until that content is copied over by hand.
# --site-vars FILE put FILE on CIDATA as playbook/site-vars.yml (the
# per-site NIC MACs; see playbook/site-vars.example.yml)
# --no-site-vars build without site vars (the first-boot playbook then
# stops until they are set on the server). One of the
# two is required; playbook/site-vars.yml in the repo
# is never copied implicitly.
# --site-vars FILE put FILE on CIDATA as playbook/site-vars.yml
# (per-site values; see playbook/site-vars.example.yml)
# --no-site-vars build without site vars (defaults; the first boot
# picks the imaging NIC by its link). One of the two
# is required; playbook/site-vars.yml in the repo is
# never copied implicitly.
# --yes do not ask for confirmation (device mode still asks
# YES for a non-removable device)
#
@@ -205,8 +205,9 @@ if [ -n "$SITE_VARS" ] && [ "$NO_SITE_VARS" -eq 1 ]; then
exit 1
fi
if [ -z "$SITE_VARS" ] && [ "$NO_SITE_VARS" -eq 0 ]; then
echo "ERROR: give --site-vars FILE (the server's NIC MACs, see playbook/site-vars.example.yml)"
echo " or --no-site-vars to build a USB whose first boot stops until they are set."
echo "ERROR: give --site-vars FILE (per-site values, see playbook/site-vars.example.yml)"
echo " or --no-site-vars (the first boot then picks the imaging NIC itself and uses"
echo " the defaults for everything else)."
exit 1
fi
if [ -n "$SITE_VARS" ]; then
@@ -214,9 +215,18 @@ if [ -n "$SITE_VARS" ]; then
echo "ERROR: --site-vars file not found: $SITE_VARS"
exit 1
fi
# Both MACs are optional: an empty pxe_nic_mac makes the first boot pick
# the imaging NIC by its link (only the imaging switch plugged in), an
# empty corp_nic_mac makes every other port a DHCP client. One that is
# set must be a MAC.
for v in pxe_nic_mac corp_nic_mac; do
if ! grep -qE "^$v:[[:space:]]*[\"']?([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}[\"']?[[:space:]]*(#.*)?$" "$SITE_VARS"; then
echo "ERROR: $SITE_VARS does not set $v to a MAC (aa:bb:cc:dd:ee:ff)"
line=$(grep -E "^$v:" "$SITE_VARS" | tail -1)
[ -n "$line" ] || continue
if grep -qE "^$v:[[:space:]]*([\"'][\"'])?[[:space:]]*(#.*)?$" <<<"$line"; then
continue
fi
if ! grep -qE "^$v:[[:space:]]*[\"']?([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}[\"']?[[:space:]]*(#.*)?$" <<<"$line"; then
echo "ERROR: $SITE_VARS sets $v to something that is not a MAC (aa:bb:cc:dd:ee:ff) or empty"
exit 1
fi
done