Files
pxe-server/autoinstall/user-data
cproudlock b11e678297 NICs: pick the imaging port at install, GE can go into any other port
Sites no longer need either MAC in site-vars.

pxe_nic_mac empty: before anything changes, the playbook runs
pxe-pick-nic --auto. It takes the one wired port with a link, accepts it
only if pxe-nic-check calls it isolated (no LLDP/CDP/802.1X, no other
DHCP server, at most 10 hosts heard in 65 s), and pins it. The choice is
saved in /etc/pxe-server/nic-pick.conf and never re-guessed at boot.
When it cannot tell (no link, two links, not isolated) nothing changes,
and the console banner says why and to run sudo pxe-pick-nic: pull the
imaging cable and plug it back in, and the port that blinked is the
imaging one. Both helpers are installed before the pick, so that works
after a refused first boot. The site's corp domains and allowed
switches are passed to the check.

corp_nic_mac empty: every other wired port (en*, eth*) is a catch-all
DHCP client, so GE can be plugged in at install or later.
- dnsmasq stays pinned with interface=pxe0. A new check refuses any
  dhcp-range outside the imaging subnet, and proxy ranges.
- The catch-all ports ignore offers from 172.16.9.1.
- UFW allows SSH and 9009 by source on any interface, and keeps those
  rules when the old any-interface rules are removed.
- pxe_nic_name must not look like en*/eth* in this mode.

pxe-netplan-apply.sh takes 3 or 5 args. Without a pinned GE NIC it:
- renames stray names (the old pxe0 after a re-pick, a leftover corp0)
  to enx<mac>, so the catch-all takes them, and restores them on revert;
- checks that every NIC that had a DHCP address before has one after,
  tracked by MAC.

With both MACs set (the WJ server), netplan, the dnsmasq pin and UFW
render byte-for-byte as before.

Tested in the pxetest VMs:
- Fresh install with no MACs and the GE link down: the imaging port was
  picked and pinned as pxe0 with 172.16.9.1, and dnsmasq bound DHCP to
  pxe0 only. GE plugged in afterwards got a lease, SSH worked, and the
  guard reported pxe0 isolated.
- --auto refuses with two links.
- The interactive pick works by pulling and replugging the link.
- A 3-arg re-apply is UNCHANGED.

Unit tests (fake sysfs) cover the re-pick and pinned-to-catch-all renames.
2026-10-16 15:56:47 -04:00

213 lines
7.7 KiB
Plaintext

#cloud-config
autoinstall:
version: 1
# Locale, keyboard, timezone
locale: en_US.UTF-8
keyboard:
layout: us
variant: ""
timezone: America/New_York
# Network: DHCP only, on every wired NIC. On the standard two-NIC box the
# GE corporate NIC gets an address and the imaging NIC (no DHCP server on
# the imaging LAN) stays without one. No static 172.16.9.1 here: matched
# by name it would land on EVERY NIC, the GE one included, and the box
# would answer as 172.16.9.1 on the corporate network. First boot's
# playbook replaces this file with the imaging NIC pinned by MAC
# (pxe_nic_mac in playbook/site-vars.yml, or picked by its link when
# empty; corp_nic_mac optional). optional: the
# install and boot never wait for a lease (air-gapped is fine).
network:
version: 2
ethernets:
any-eth:
match:
name: "en*"
dhcp4: true
dhcp6: false
optional: true
# The install is always offline, even when the corp NIC has a lease and
# internet: the mirror points at a closed local port, so subiquity's
# mirror check fails and it installs from the ISO alone (no "security
# updates" step from the archive). The base is then exactly the ISO's,
# which is what download-packages.sh resolved the offline set against;
# late-commands patch it from CIDATA. The real archive sources are
# written back at the end of late-commands.
apt:
geoip: false
preserve_sources_list: false
primary:
- arches: [default]
uri: "http://127.0.0.1:9/ubuntu"
fallback: offline-install
# Storage configuration
storage:
layout:
name: lvm
match:
size: largest
swap:
size: 0
# User identity
identity:
hostname: pxeserver
username: pxe
password: '$6$MV3UwfSxKqxDYCwt$2IPUfSvqpzUdH6QVKHfPH8I9E0SPqNCh0lLvn.D9btxhZUYmceOGcB51Ryfg1Fojlroin.qguSKYFoIDGUIC20'
# Enable SSH
ssh:
install-server: true
allow-pw: true
# Installer-stage late commands
late-commands:
# Step 1: Upgrade and install from the offline set on CIDATA
# (packages/ is an apt repo: debs + Packages + explicit.list). Any apt
# error fails the install; the log is /var/log/offline-apt-install.log.
- |
curtin in-target --target=/target -- bash -euc '
mkdir -p /mnt/cidata
CIDATA_DEV=$(blkid -L CIDATA)
mount -o ro "$CIDATA_DEV" /mnt/cidata
set +e
bash /mnt/cidata/offline-apt-install.sh /mnt/cidata/packages 2>&1 | tee /var/log/offline-apt-install.log
rc=${PIPESTATUS[0]}
set -e
umount /mnt/cidata
if [ "$rc" -ne 0 ]; then
echo "offline package install FAILED (rc=$rc), see /var/log/offline-apt-install.log" >&2
exit "$rc"
fi
if command -v nmcli >/dev/null; then
systemctl enable NetworkManager
fi
'
# Step 2: Disable networkd-wait-online (hangs on air-gapped network)
- ln -sf /dev/null /target/etc/systemd/system/systemd-networkd-wait-online.service
# Step 3: Create first-boot.sh. rc.local runs it on every boot until the
# playbook succeeds (at most MAX attempts). A failure leaves
# /var/lib/pxe-firstboot.failed and a banner on the console (/etc/issue)
# and at SSH login (/etc/motd) pointing at /var/log/first-boot.log.
- |
curtin in-target --target=/target -- bash -c '
cat <<"EOF" > /opt/first-boot.sh
#!/bin/bash
MAX=5
STATE=/var/lib/pxe-firstboot
FAILED=/var/lib/pxe-firstboot.failed
mkdir -p "$STATE"
N=$(( $(cat "$STATE/attempts" 2>/dev/null || echo 0) + 1 ))
echo "$N" > "$STATE/attempts"
echo "=== first-boot attempt $N of $MAX, $(date -Is) ==="
banner_off() {
for f in /etc/issue /etc/motd; do
[ -f "$f" ] && sed -i "/^### PXE FIRST-BOOT BEGIN/,/^### PXE FIRST-BOOT END/d" "$f"
done
}
banner_on() {
banner_off
for f in /etc/issue /etc/motd; do
{
echo "### PXE FIRST-BOOT BEGIN"
echo "!!! PXE server setup (first-boot playbook) FAILED: $1"
echo "!!! Log: /var/log/first-boot.log Marker: $FAILED"
echo "!!! $2"
echo "### PXE FIRST-BOOT END"
} >> "$f"
done
}
# Expand root LV to full disk BEFORE playbook (playbook copies ~130GB of drivers+PPKGs)
ROOT_DEV=$(findmnt -n -o SOURCE /)
ROOT_DISK=$(lsblk -n -o PKNAME "$(readlink -f "$ROOT_DEV")" | tail -1)
PV_PART=$(pvs --noheadings -o pv_name 2>/dev/null | tr -d "[:space:]" | head -1)
if [ -n "$ROOT_DISK" ] && [ -n "$PV_PART" ]; then
PART_NUM=$(echo "$PV_PART" | grep -o "[0-9]*$")
growpart "/dev/${ROOT_DISK}" "${PART_NUM}" 2>&1 || true
pvresize "$PV_PART" 2>&1 || true
fi
lvextend -r -l +100%FREE /dev/mapper/ubuntu--vg-ubuntu--lv 2>&1 || true
RC=1
WHY="CIDATA (the installer USB) not found"
CIDATA_DEV=$(blkid -L CIDATA)
if [ -n "$CIDATA_DEV" ]; then
mkdir -p /mnt/usb
if mount "$CIDATA_DEV" /mnt/usb; then
if [ -f /mnt/usb/playbook/pxe_server_setup.yml ]; then
cd /mnt/usb/playbook
rm -f "$STATE/nic-hint"
ansible-playbook -i localhost, -c local pxe_server_setup.yml
RC=$?
WHY="ansible-playbook exit code $RC"
# pxe-pick-nic leaves the reason it could not pick the imaging NIC
[ "$RC" -ne 0 ] && [ -s "$STATE/nic-hint" ] && WHY="$(head -1 "$STATE/nic-hint")"
cd /
else
WHY="no playbook/pxe_server_setup.yml on CIDATA"
fi
umount /mnt/usb
else
WHY="could not mount CIDATA ($CIDATA_DEV)"
fi
fi
if [ "$RC" -eq 0 ]; then
echo "=== first-boot: playbook succeeded on attempt $N ==="
rm -f "$FAILED"
banner_off
sed -i "s|^/opt/first-boot.sh.*|# &|" /etc/rc.local
exit 0
fi
echo "=== first-boot: FAILED ($WHY), attempt $N of $MAX ==="
printf "attempt=%s of %s\nwhen=%s\nreason=%s\nlog=/var/log/first-boot.log\n" "$N" "$MAX" "$(date -Is)" "$WHY" > "$FAILED"
if [ "$N" -ge "$MAX" ]; then
sed -i "s|^/opt/first-boot.sh.*|# &|" /etc/rc.local
banner_on "$WHY (attempt $N of $MAX)" "Gave up retrying. Fix it, then run: sudo /opt/first-boot.sh after rm $STATE/attempts"
else
banner_on "$WHY (attempt $N of $MAX)" "It runs again at the next boot (fix the cause first, e.g. playbook/site-vars.yml on the USB)."
fi
exit "$RC"
EOF
chmod +x /opt/first-boot.sh
'
# Step 4: Create rc.local (the log is appended to, one block per attempt)
- |
curtin in-target --target=/target -- bash -c '
cat <<"EOF" > /etc/rc.local
#!/bin/bash
/opt/first-boot.sh >> /var/log/first-boot.log 2>&1 &
exit 0
EOF
chmod +x /etc/rc.local
'
# Step 5: Real archive sources for later online maintenance (the
# install used a dead mirror on purpose, see apt: above).
- |
rm -f /target/etc/apt/sources.list.d/*.list /target/etc/apt/sources.list.d/*.curtin.orig
echo "# Ubuntu sources have moved to /etc/apt/sources.list.d/ubuntu.sources" > /target/etc/apt/sources.list
cat > /target/etc/apt/sources.list.d/ubuntu.sources <<"EOF"
Types: deb
URIs: http://archive.ubuntu.com/ubuntu/
Suites: noble noble-updates noble-backports
Components: main restricted universe multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
Types: deb
URIs: http://security.ubuntu.com/ubuntu/
Suites: noble-security
Components: main restricted universe multiverse
Signed-By: /usr/share/keyrings/ubuntu-archive-keyring.gpg
EOF
user-data:
disable_root: false
refresh-installer:
update: no