NICs: pick the imaging port at install, GE can go into any other port
Sites no longer need either MAC in site-vars. pxe_nic_mac empty: before anything changes, the playbook runs pxe-pick-nic --auto. It takes the one wired port with a link, accepts it only if pxe-nic-check calls it isolated (no LLDP/CDP/802.1X, no other DHCP server, at most 10 hosts heard in 65 s), and pins it. The choice is saved in /etc/pxe-server/nic-pick.conf and never re-guessed at boot. When it cannot tell (no link, two links, not isolated) nothing changes, and the console banner says why and to run sudo pxe-pick-nic: pull the imaging cable and plug it back in, and the port that blinked is the imaging one. Both helpers are installed before the pick, so that works after a refused first boot. The site's corp domains and allowed switches are passed to the check. corp_nic_mac empty: every other wired port (en*, eth*) is a catch-all DHCP client, so GE can be plugged in at install or later. - dnsmasq stays pinned with interface=pxe0. A new check refuses any dhcp-range outside the imaging subnet, and proxy ranges. - The catch-all ports ignore offers from 172.16.9.1. - UFW allows SSH and 9009 by source on any interface, and keeps those rules when the old any-interface rules are removed. - pxe_nic_name must not look like en*/eth* in this mode. pxe-netplan-apply.sh takes 3 or 5 args. Without a pinned GE NIC it: - renames stray names (the old pxe0 after a re-pick, a leftover corp0) to enx<mac>, so the catch-all takes them, and restores them on revert; - checks that every NIC that had a DHCP address before has one after, tracked by MAC. With both MACs set (the WJ server), netplan, the dnsmasq pin and UFW render byte-for-byte as before. Tested in the pxetest VMs: - Fresh install with no MACs and the GE link down: the imaging port was picked and pinned as pxe0 with 172.16.9.1, and dnsmasq bound DHCP to pxe0 only. GE plugged in afterwards got a lease, SSH worked, and the guard reported pxe0 isolated. - --auto refuses with two links. - The interactive pick works by pulling and replugging the link. - A 3-arg re-apply is UNCHANGED. Unit tests (fake sysfs) cover the re-pick and pinned-to-catch-all renames.
This commit is contained in:
cproudlock
12
SETUP.md
12
SETUP.md
@@ -144,9 +144,15 @@ This creates a bootable USB with:
|
||||
|
||||
### Step 5: Connect to Isolated Network
|
||||
|
||||
Both NICs are cabled before the install: the imaging NIC (`pxe_nic_mac`)
|
||||
to the isolated switch for PXE clients, the corp NIC (`corp_nic_mac`) to
|
||||
the GE network. Nothing needs to move after the install.
|
||||
During the install plug in ONLY the imaging switch (GE unplugged). With
|
||||
`pxe_nic_mac` empty in site-vars, the first boot picks the one wired port
|
||||
with a link, checks it is isolated (no GE switch, no 802.1X, no other DHCP
|
||||
server) and pins it. Then plug the GE cable into any other port: it becomes
|
||||
a DHCP client for SSH and the webapp, and dnsmasq never serves it. If the
|
||||
first boot cannot tell which port is the imaging one, the console says so:
|
||||
log in and run `sudo pxe-pick-nic`, pull the imaging cable and plug it back
|
||||
in. Sites that prefer to pin both ports by MAC set `pxe_nic_mac` and
|
||||
`corp_nic_mac` instead; both cables can then be in from the start.
|
||||
|
||||
### Step 6: Import WinPE Content (if not bundled in Step 3)
|
||||
|
||||
|
||||
@@ -14,8 +14,9 @@ autoinstall:
|
||||
# the imaging LAN) stays without one. No static 172.16.9.1 here: matched
|
||||
# by name it would land on EVERY NIC, the GE one included, and the box
|
||||
# would answer as 172.16.9.1 on the corporate network. First boot's
|
||||
# playbook replaces this file with the two-NIC config pinned by MAC
|
||||
# (pxe_nic_mac / corp_nic_mac in playbook/site-vars.yml). optional: the
|
||||
# playbook replaces this file with the imaging NIC pinned by MAC
|
||||
# (pxe_nic_mac in playbook/site-vars.yml, or picked by its link when
|
||||
# empty; corp_nic_mac optional). optional: the
|
||||
# install and boot never wait for a lease (air-gapped is fine).
|
||||
network:
|
||||
version: 2
|
||||
@@ -139,9 +140,12 @@ autoinstall:
|
||||
if mount "$CIDATA_DEV" /mnt/usb; then
|
||||
if [ -f /mnt/usb/playbook/pxe_server_setup.yml ]; then
|
||||
cd /mnt/usb/playbook
|
||||
rm -f "$STATE/nic-hint"
|
||||
ansible-playbook -i localhost, -c local pxe_server_setup.yml
|
||||
RC=$?
|
||||
WHY="ansible-playbook exit code $RC"
|
||||
# pxe-pick-nic leaves the reason it could not pick the imaging NIC
|
||||
[ "$RC" -ne 0 ] && [ -s "$STATE/nic-hint" ] && WHY="$(head -1 "$STATE/nic-hint")"
|
||||
cd /
|
||||
else
|
||||
WHY="no playbook/pxe_server_setup.yml on CIDATA"
|
||||
|
||||
@@ -20,21 +20,38 @@ contribute a `config/sites/<sitename>.yaml` template back to the repo.
|
||||
|
||||
### Network
|
||||
|
||||
Every server has two NICs: the imaging NIC (isolated imaging LAN, static
|
||||
172.16.9.1, the only NIC that serves DHCP/TFTP/HTTP boot/SMB) and the GE
|
||||
corporate NIC (DHCP client; SSH and the webapp on 9009 only). Put both MACs
|
||||
in `playbook/site-vars.yml` (copy `playbook/site-vars.example.yml`) before
|
||||
building the USB: `pxe_nic_mac`, `corp_nic_mac`. The installer brings the
|
||||
network up with DHCP only; the first-boot playbook stops before changing
|
||||
anything if either MAC is unset or missing, then pins both NICs by MAC in
|
||||
netplan (names `pxe0` / `corp0`, overridable), dnsmasq, UFW and Samba.
|
||||
The imaging NIC (isolated imaging LAN, static 172.16.9.1, the only NIC that
|
||||
serves DHCP/TFTP/HTTP boot/SMB) is pinned by MAC. The GE corporate side (DHCP
|
||||
client; SSH and the webapp on 9009 only) is optional. Both MACs in
|
||||
`playbook/site-vars.yml` (copy `playbook/site-vars.example.yml`) may be left
|
||||
empty:
|
||||
|
||||
- `pxe_nic_mac` empty: the first boot picks the imaging NIC itself
|
||||
(`pxe-pick-nic --auto`). It takes the one wired port with a link, and only
|
||||
if `pxe-nic-check` calls it isolated, then pins it; the choice is in
|
||||
`/etc/pxe-server/nic-pick.conf` and is never re-guessed at boot. So during
|
||||
the install plug in ONLY the imaging switch. When it cannot tell (no link,
|
||||
two links, or that port sees GE) it changes nothing and the console says
|
||||
to run `sudo pxe-pick-nic`: pull the imaging cable and plug it back in,
|
||||
and the port that blinked is the imaging one. Run it again after swapping
|
||||
a dead USB NIC, then re-run the playbook.
|
||||
- `corp_nic_mac` empty: every other wired port (`en*`, `eth*`) is a DHCP
|
||||
client, so the GE cable can go into any of them, at install or later.
|
||||
dnsmasq is pinned to the imaging NIC (`interface=`) and serves only the
|
||||
imaging subnet, and those ports ignore DHCP offers from 172.16.9.1. UFW
|
||||
allows SSH and 9009 by source, not by interface.
|
||||
- `corp_nic_mac` set: that port alone is the GE NIC, named `corp0` (as on
|
||||
the WJ server).
|
||||
|
||||
The first-boot playbook pins the result in netplan (names `pxe0` / `corp0`,
|
||||
overridable), dnsmasq, UFW and Samba.
|
||||
|
||||
| Value | Default | Where it lives |
|
||||
|-------------------|----------------------|--------------------------------------------------------------------------------|
|
||||
| PXE server IP | 172.16.9.1 | `playbook/pxe_server_setup.yml` (dnsmasq config, iPXE script, samba conf, webapp env), `playbook/startnet.cmd` (mount paths), `boot-tools/blancco/grub-blancco.cfg` (TFTP/HTTP URLs) |
|
||||
| PXE subnet | 172.16.9.0/24 | Same as above, plus `playbook/pxe_server_setup.yml` (UFW rules) |
|
||||
| DHCP range | 172.16.9.10-100 | `playbook/site-vars.yml` (`pxe_dhcp_start`, `pxe_dhcp_end`) |
|
||||
| NIC MACs / names | (required) / pxe0, corp0 | `playbook/site-vars.yml` (`pxe_nic_mac`, `corp_nic_mac`, `pxe_nic_name`, `corp_nic_name`) |
|
||||
| NIC MACs / names | (picked at install) / pxe0, corp0 | `playbook/site-vars.yml` (`pxe_nic_mac`, `corp_nic_mac`, `pxe_nic_name`, `corp_nic_name`) |
|
||||
| Hostname | pxeserver | `autoinstall/user-data` (identity.hostname) |
|
||||
|
||||
### Access from the GE corporate network
|
||||
@@ -46,7 +63,7 @@ warning on every run while that is so. Set these in `playbook/site-vars.yml`:
|
||||
|
||||
| Var | Default | Effect |
|
||||
|--------------------------|---------|--------|
|
||||
| `corp_admin_cidrs` | `[]` | List of admin subnets/hosts, e.g. `["10.134.48.0/24", "3.12.34.56/32"]`. When non-empty, UFW allows 22 and 9009 on `corp_nic_name` from these only and drops the any-source rules. The play refuses (nothing changed) if an entry is invalid or your own SSH session is not inside one of them. The imaging NIC keeps its rules. |
|
||||
| `corp_admin_cidrs` | `[]` | List of admin subnets/hosts, e.g. `["10.134.48.0/24", "3.12.34.56/32"]`. When non-empty, UFW allows 22 and 9009 from these only (on `corp_nic_name` when `corp_nic_mac` is set, else on any interface) and drops the any-source rules. The play refuses (nothing changed) if an entry is invalid or your own SSH session is not inside one of them. The imaging NIC keeps its rules. |
|
||||
| `corp_ssh_password_auth` | `true` | `false` adds `/etc/ssh/sshd_config.d/10-pxe-corp-auth.conf`: `PasswordAuthentication no`, re-allowed by a `Match Address` block for `pxe_subnet` and localhost. Keys only from GE. The play refuses while `pxe` has no `~/.ssh/authorized_keys`, and removes the drop-in again if `sshd -t` rejects it. |
|
||||
| `allow_disruptive` | `false` | Re-runs while imaging is active (SMB sessions on 445, or DHCP leases granted in the last `pxe_busy_window_min` = 30 min) do not bounce the NICs or restart dnsmasq/Apache/Samba; a needed network change stops the play. `-e allow_disruptive=true` overrides. |
|
||||
|
||||
@@ -58,8 +75,9 @@ desks' subnet, then (after `ssh-copy-id`) `corp_ssh_password_auth: false`.
|
||||
The network step refuses before changing anything when the NIC that
|
||||
`pxe_nic_mac` names holds the default route, a DHCP lease, or the route to
|
||||
your SSH client (swapped MACs or cables). When it does apply, it arms a
|
||||
120 s auto-revert timer (`pxe-netplan-revert-*`), checks both names, the
|
||||
imaging address and that the corp NIC gets its IPv4 back within 30 s, and
|
||||
120 s auto-revert timer (`pxe-netplan-revert-*`), checks the names, the
|
||||
imaging address and that the corp NIC (or, without one pinned, every other
|
||||
port that had an address) gets its IPv4 back within 30 s, and
|
||||
restores the previous netplan config AND NIC names on any failure. Backups:
|
||||
`/var/backups/netplan-<timestamp>/`.
|
||||
|
||||
|
||||
@@ -32,7 +32,11 @@ Sources: `playbook/pxe-server-helpers/`, installed by `pxe_server_setup.yml`
|
||||
`pxe_nic_allowed_switches`; `pxe_imaging_mac` / `pxe_ge_mac` are derived
|
||||
from the first two). The same MACs pin the NICs in netplan
|
||||
(`playbook/templates/netplan-pxe.yaml.j2`, names `pxe0` / `corp0`) and in
|
||||
`/etc/dnsmasq.d/pxe-nic-only.conf`.
|
||||
`/etc/dnsmasq.d/pxe-nic-only.conf`. With `pxe_nic_mac` empty the playbook
|
||||
picks the imaging NIC at install time with `pxe-pick-nic --auto`, which
|
||||
runs this check on the one port with a link (`--iface X --probe`, 65 s)
|
||||
and accepts only "isolated". With `corp_nic_mac` empty `ge_mac` is blank
|
||||
and the GE NIC is whichever other port holds a DHCP lease.
|
||||
|
||||
## How it decides
|
||||
|
||||
|
||||
@@ -1,12 +1,17 @@
|
||||
#!/bin/bash
|
||||
# pxe-netplan-apply.sh - install the two-NIC netplan config without ever
|
||||
# leaving the box unreachable on the corporate NIC.
|
||||
# pxe-netplan-apply.sh - install the PXE server's netplan config without
|
||||
# ever leaving the box unreachable on the corporate side.
|
||||
#
|
||||
# Usage:
|
||||
# pxe-netplan-apply.sh STAGED_YAML PXE_NAME PXE_MAC CORP_NAME CORP_MAC
|
||||
# pxe-netplan-apply.sh --preflight STAGED_YAML PXE_NAME PXE_MAC CORP_NAME CORP_MAC
|
||||
# pxe-netplan-apply.sh STAGED_YAML PXE_NAME PXE_MAC [CORP_NAME CORP_MAC]
|
||||
# pxe-netplan-apply.sh --preflight STAGED_YAML PXE_NAME PXE_MAC [CORP_NAME CORP_MAC]
|
||||
# pxe-netplan-apply.sh --revert BACKUP_DIR (the auto-revert timer)
|
||||
#
|
||||
# Without CORP_NAME/CORP_MAC (corp_nic_mac empty in the playbook) only the
|
||||
# imaging NIC is pinned and renamed; the other wired ports are catch-all
|
||||
# DHCP clients that keep their names. Every pre-flight refusal about the
|
||||
# imaging NIC applies the same.
|
||||
#
|
||||
# Env (set by pxe_server_setup.yml):
|
||||
# PXE_SUBNET imaging LAN, e.g. 172.16.9.0/24 (SSH peers inside it may
|
||||
# legitimately come in on the imaging NIC)
|
||||
@@ -128,14 +133,16 @@ ssh_peers() {
|
||||
preflight() {
|
||||
local pif cif peer dev bad=0
|
||||
pif=$(iface_of "$PXE_MAC")
|
||||
cif=$(iface_of "$CORP_MAC")
|
||||
if [ -z "$pif" ]; then
|
||||
log "REFUSED: no physical NIC has pxe_nic_mac $PXE_MAC"
|
||||
return 1
|
||||
fi
|
||||
if [ -z "$cif" ]; then
|
||||
log "REFUSED: no physical NIC has corp_nic_mac $CORP_MAC"
|
||||
return 1
|
||||
if [ -n "$CORP_MAC" ]; then
|
||||
cif=$(iface_of "$CORP_MAC")
|
||||
if [ -z "$cif" ]; then
|
||||
log "REFUSED: no physical NIC has corp_nic_mac $CORP_MAC"
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
if default_route_devs | grep -qxF "$pif"; then
|
||||
log "REFUSED: $pif (pxe_nic_mac $PXE_MAC) holds the default route"
|
||||
@@ -156,8 +163,8 @@ preflight() {
|
||||
bad=1
|
||||
done < <(ssh_peers)
|
||||
if [ "$bad" -ne 0 ]; then
|
||||
log "The NIC named by pxe_nic_mac looks like the GE corporate one: pxe_nic_mac"
|
||||
log "and corp_nic_mac are probably swapped, or the cables are. It would become"
|
||||
log "The NIC named by pxe_nic_mac looks like the GE corporate one: the MACs are"
|
||||
log "probably swapped, or the cables are. It would become"
|
||||
log "static ${PXE_SUBNET:-imaging} with no gateway and cut this box off. Nothing was changed."
|
||||
log "NICs now: $(for d in "$SYSFS_NET"/*; do [ -e "$d/device" ] && printf '%s=%s ' "$(basename "$d")" "$(cat "$d/address" 2>/dev/null)"; done)"
|
||||
return 1
|
||||
@@ -181,32 +188,38 @@ link_rename() {
|
||||
return 1
|
||||
}
|
||||
|
||||
# set_names MAC1 NAME1 MAC2 NAME2: give each MAC its name. A NIC that must
|
||||
# set_names MAC1 NAME1 [MAC2 NAME2]: give each MAC its name. A NIC that must
|
||||
# change goes through a temporary name first, so a swap (pxe0 <-> corp0)
|
||||
# works in both directions. Temporary names carry this run's PID, so they
|
||||
# never collide with a name left over from an earlier (failed) pass.
|
||||
# never collide with a name left over from an earlier (failed) pass. A pair
|
||||
# with an empty MAC or name is skipped (no pinned GE NIC).
|
||||
set_names() {
|
||||
local -a macs=("$1" "$3") want=("$2" "$4") cur=()
|
||||
local -a macs=() want=() cur=()
|
||||
local i
|
||||
while [ $# -ge 2 ]; do
|
||||
if [ -n "$1" ] && [ -n "$2" ]; then
|
||||
macs+=("$1"); want+=("$2")
|
||||
fi
|
||||
shift 2
|
||||
done
|
||||
RENAME_SEQ=$(( ${RENAME_SEQ:-0} + 1 ))
|
||||
for i in 0 1; do
|
||||
[ -n "${want[i]}" ] || return 0
|
||||
for i in "${!macs[@]}"; do
|
||||
cur[i]=$(iface_of "${macs[i]}")
|
||||
if [ -z "${cur[i]}" ]; then
|
||||
log "ERROR: no NIC with MAC ${macs[i]}"
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
for i in 0 1; do
|
||||
for i in "${!macs[@]}"; do
|
||||
[ "${cur[i]}" = "${want[i]}" ] && continue
|
||||
link_rename "${cur[i]}" "pxt$$-$RENAME_SEQ$i" || return 1
|
||||
cur[i]=pxt$$-$RENAME_SEQ$i
|
||||
done
|
||||
for i in 0 1; do
|
||||
for i in "${!macs[@]}"; do
|
||||
[ "${cur[i]}" = "${want[i]}" ] && continue
|
||||
link_rename "${cur[i]}" "${want[i]}" || return 1
|
||||
done
|
||||
for i in 0 1; do
|
||||
for i in "${!macs[@]}"; do
|
||||
if [ "$(iface_of "${macs[i]}")" != "${want[i]}" ]; then
|
||||
log "ERROR: MAC ${macs[i]} is not named ${want[i]} after renaming"
|
||||
return 1
|
||||
@@ -216,8 +229,49 @@ set_names() {
|
||||
}
|
||||
|
||||
names_ok() {
|
||||
[ "$(iface_of "$PXE_MAC")" = "$PXE_NAME" ] &&
|
||||
[ "$(iface_of "$PXE_MAC")" = "$PXE_NAME" ] || return 1
|
||||
if [ -n "$CORP_MAC" ]; then
|
||||
[ "$(iface_of "$CORP_MAC")" = "$CORP_NAME" ]
|
||||
else
|
||||
[ -z "$(stray_names)" ]
|
||||
fi
|
||||
}
|
||||
|
||||
# No pinned GE NIC: "MAC NAME" for every other physical NIC whose name the
|
||||
# catch-all (en*, eth*) would not match - the previous imaging NIC still
|
||||
# called pxe0 after a re-pick, or corp0 from a pinned config. Each gets the
|
||||
# kernel-style MAC name enx<mac>, so it becomes a catch-all DHCP client and
|
||||
# pxe0 is free for the new imaging NIC.
|
||||
stray_names() {
|
||||
local d n m
|
||||
[ -z "$CORP_MAC" ] || return 0
|
||||
for d in "$SYSFS_NET"/*; do
|
||||
[ -e "$d/device" ] || continue
|
||||
n=${d##*/}
|
||||
m=$(cat "$d/address" 2>/dev/null)
|
||||
[ "$m" = "$PXE_MAC" ] && continue
|
||||
[ "$(cat "$d/type" 2>/dev/null)" = 1 ] || continue
|
||||
[ -e "$d/wireless" ] || [ -e "$d/phy80211" ] || [ -e "$d/bridge" ] && continue
|
||||
case "$n" in en*|eth*) continue ;; esac
|
||||
echo "$m enx${m//:/}"
|
||||
done
|
||||
return 0
|
||||
}
|
||||
|
||||
# MACs of the physical NICs other than the imaging one that hold a DHCP
|
||||
# (dynamic) IPv4: the GE side. They must have one again after the apply,
|
||||
# or an admin on the GE side may have been cut off. Static addresses (the
|
||||
# previous imaging NIC's 172.16.9.1) do not count. By MAC: names change.
|
||||
other_v4_macs() {
|
||||
local d n m
|
||||
for d in "$SYSFS_NET"/*; do
|
||||
[ -e "$d/device" ] || continue
|
||||
n=${d##*/}
|
||||
m=$(cat "$d/address" 2>/dev/null)
|
||||
[ "$m" = "$PXE_MAC" ] && continue
|
||||
ip -4 -o addr show dev "$n" scope global dynamic 2>/dev/null | grep -q ' inet ' && echo "$m"
|
||||
done
|
||||
return 0
|
||||
}
|
||||
|
||||
# --- backup / restore --------------------------------------------------------
|
||||
@@ -239,6 +293,7 @@ write_state() {
|
||||
echo "ORIG_PXE_IF=$ORIG_PXE_IF"
|
||||
echo "CORP_MAC=$CORP_MAC"
|
||||
echo "ORIG_CORP_IF=$ORIG_CORP_IF"
|
||||
echo "EXTRA=$EXTRA"
|
||||
} >"$BACKUP.state"
|
||||
}
|
||||
|
||||
@@ -249,6 +304,7 @@ read_state() {
|
||||
NETPLAN_DIR) [[ "$v" =~ ^/[A-Za-z0-9/._-]*$ ]] && NETPLAN_DIR=$v ;;
|
||||
PXE_MAC|CORP_MAC) [[ "$v" =~ ^([0-9a-f]{2}:){5}[0-9a-f]{2}$ ]] && printf -v "$k" '%s' "$v" ;;
|
||||
ORIG_PXE_IF|ORIG_CORP_IF) [[ "$v" =~ ^[A-Za-z0-9._-]{0,15}$ ]] && printf -v "$k" '%s' "$v" ;;
|
||||
EXTRA) [[ "$v" =~ ^(([0-9a-f]{2}:){5}[0-9a-f]{2}=[A-Za-z0-9._-]{1,15},?)*$ ]] && EXTRA=$v ;;
|
||||
esac
|
||||
done <"$BACKUP.state"
|
||||
TARGET=$NETPLAN_DIR/50-cloud-init.yaml
|
||||
@@ -268,8 +324,11 @@ restore() {
|
||||
log "restoring the previous config from $BACKUP"
|
||||
find "$NETPLAN_DIR" -maxdepth 1 -type f \( -name '*.yaml' -o -name '*.yml' \) -delete
|
||||
cp -a "$BACKUP/." "$NETPLAN_DIR/"
|
||||
if [ -n "${ORIG_PXE_IF:-}" ] && [ -n "${ORIG_CORP_IF:-}" ]; then
|
||||
set_names "$PXE_MAC" "$ORIG_PXE_IF" "$CORP_MAC" "$ORIG_CORP_IF" ||
|
||||
if [ -n "${ORIG_PXE_IF:-}" ]; then
|
||||
local -a back=()
|
||||
local pair
|
||||
for pair in ${EXTRA//,/ }; do back+=("${pair%%=*}" "${pair#*=}"); done
|
||||
set_names "$PXE_MAC" "$ORIG_PXE_IF" "$CORP_MAC" "${ORIG_CORP_IF:-}" "${back[@]}" ||
|
||||
{ log "ERROR: could not put the previous NIC names back"; rc=1; }
|
||||
fi
|
||||
netplan generate && netplan apply || { log "ERROR: netplan apply of the restored config failed"; rc=1; }
|
||||
@@ -300,7 +359,7 @@ case "${1:-}" in
|
||||
log "revert: $BACKUP already committed or reverted; nothing to do"
|
||||
exit 0
|
||||
fi
|
||||
PXE_MAC="" CORP_MAC="" ORIG_PXE_IF="" ORIG_CORP_IF=""
|
||||
PXE_MAC="" CORP_MAC="" ORIG_PXE_IF="" ORIG_CORP_IF="" EXTRA=""
|
||||
read_state
|
||||
log "revert: the apply was not confirmed within its window"
|
||||
restore
|
||||
@@ -308,15 +367,16 @@ case "${1:-}" in
|
||||
;;
|
||||
esac
|
||||
|
||||
if [ $# -ne 5 ]; then
|
||||
echo "usage: $0 [--preflight] STAGED_YAML PXE_NAME PXE_MAC CORP_NAME CORP_MAC" >&2
|
||||
if [ $# -ne 3 ] && [ $# -ne 5 ]; then
|
||||
echo "usage: $0 [--preflight] STAGED_YAML PXE_NAME PXE_MAC [CORP_NAME CORP_MAC]" >&2
|
||||
exit 2
|
||||
fi
|
||||
STAGED=$1
|
||||
PXE_NAME=$2
|
||||
PXE_MAC=${3,,}
|
||||
CORP_NAME=$4
|
||||
CORP_MAC=${5,,}
|
||||
CORP_NAME=${4:-}
|
||||
CORP_MAC=${5:-}
|
||||
CORP_MAC=${CORP_MAC,,}
|
||||
|
||||
# 1. validate in a scratch root
|
||||
SCRATCH=$(mktemp -d)
|
||||
@@ -330,7 +390,7 @@ fi
|
||||
|
||||
# 2. already in place?
|
||||
if cmp -s "$STAGED" "$TARGET" && [ -z "$(other_files)" ] && names_ok; then
|
||||
log "netplan already in place, $PXE_NAME and $CORP_NAME present"
|
||||
log "netplan already in place, $PXE_NAME${CORP_NAME:+ and $CORP_NAME} present"
|
||||
echo UNCHANGED
|
||||
exit 0
|
||||
fi
|
||||
@@ -350,14 +410,26 @@ flock -w 60 9 || { log "ERROR: another pxe-netplan-apply holds $LOCK_FILE; nothi
|
||||
|
||||
# 4. back up, record names, arm the auto-revert
|
||||
ORIG_PXE_IF=$(iface_of "$PXE_MAC")
|
||||
ORIG_CORP_IF=$(iface_of "$CORP_MAC")
|
||||
ORIG_CORP_IF=""
|
||||
[ -n "$CORP_MAC" ] && ORIG_CORP_IF=$(iface_of "$CORP_MAC")
|
||||
CORP_HAD_V4=0
|
||||
has_v4 "$ORIG_CORP_IF" && CORP_HAD_V4=1
|
||||
[ -n "$ORIG_CORP_IF" ] && has_v4 "$ORIG_CORP_IF" && CORP_HAD_V4=1
|
||||
OTHERS_V4=""
|
||||
[ -z "$CORP_MAC" ] && OTHERS_V4=$(other_v4_macs)
|
||||
# NICs to move off a name the catch-all would not match, recorded with
|
||||
# their current names for the revert ("mac=name,...").
|
||||
EXTRA=""
|
||||
FREE=()
|
||||
while read -r m newname; do
|
||||
[ -n "$m" ] || continue
|
||||
EXTRA="${EXTRA:+$EXTRA,}$m=$(iface_of "$m")"
|
||||
FREE+=("$m" "$newname")
|
||||
done < <(stray_names)
|
||||
BACKUP=$BACKUP_ROOT/netplan-$(date +%Y%m%d-%H%M%S)
|
||||
mkdir -p "$BACKUP"
|
||||
cp -a "$NETPLAN_DIR/." "$BACKUP/"
|
||||
write_state
|
||||
log "previous config saved in $BACKUP (NICs: $ORIG_PXE_IF, $ORIG_CORP_IF)"
|
||||
log "previous config saved in $BACKUP (NICs: $ORIG_PXE_IF${ORIG_CORP_IF:+, $ORIG_CORP_IF})"
|
||||
|
||||
REVERT_UNIT=""
|
||||
if [ "$REVERT_AFTER" -gt 0 ] 2>/dev/null && command -v systemd-run >/dev/null; then
|
||||
@@ -389,20 +461,32 @@ netplan generate || fail "netplan generate failed"
|
||||
|
||||
# 6. stable names now, so dnsmasq/UFW/Samba (which use them) work before
|
||||
# the next reboot. Does not depend on the netplan version renaming.
|
||||
set_names "$PXE_MAC" "$PXE_NAME" "$CORP_MAC" "$CORP_NAME" || fail "NIC rename failed"
|
||||
[ ${#FREE[@]} -gt 0 ] && log "freeing names for the catch-all: $EXTRA"
|
||||
set_names "$PXE_MAC" "$PXE_NAME" "$CORP_MAC" "$CORP_NAME" "${FREE[@]}" || fail "NIC rename failed"
|
||||
|
||||
# 7. apply
|
||||
netplan apply || fail "netplan apply failed"
|
||||
|
||||
# 8. check
|
||||
names_ok || fail "$PXE_NAME / $CORP_NAME are not on MACs $PXE_MAC / $CORP_MAC after apply"
|
||||
names_ok || fail "$PXE_NAME${CORP_NAME:+ / $CORP_NAME} not on MAC $PXE_MAC${CORP_MAC:+ / $CORP_MAC} after apply"
|
||||
ok=0
|
||||
for _ in $(seq 1 "$PXE_WAIT"); do
|
||||
if has_v4 "$PXE_NAME"; then ok=1; break; fi
|
||||
sleep 1
|
||||
done
|
||||
[ "$ok" -eq 1 ] || fail "$PXE_NAME has no IPv4 address after ${PXE_WAIT}s"
|
||||
if [ "$CORP_HAD_V4" -eq 1 ]; then
|
||||
for m in $OTHERS_V4; do
|
||||
ok=0
|
||||
for _ in $(seq 1 "$CORP_WAIT"); do
|
||||
n=$(iface_of "$m")
|
||||
if [ -n "$n" ] && has_v4 "$n"; then ok=1; break; fi
|
||||
sleep 1
|
||||
done
|
||||
[ "$ok" -eq 1 ] || fail "the NIC with MAC $m had a DHCP IPv4 before and has none after ${CORP_WAIT}s"
|
||||
done
|
||||
if [ -z "$CORP_MAC" ]; then
|
||||
:
|
||||
elif [ "$CORP_HAD_V4" -eq 1 ]; then
|
||||
ok=0
|
||||
for _ in $(seq 1 "$CORP_WAIT"); do
|
||||
if has_v4 "$CORP_NAME"; then ok=1; break; fi
|
||||
@@ -427,6 +511,6 @@ touch "$BACKUP.committed"
|
||||
cancel_revert
|
||||
trap - INT TERM
|
||||
ip -br addr show dev "$PXE_NAME" 2>/dev/null
|
||||
ip -br addr show dev "$CORP_NAME" 2>/dev/null
|
||||
[ -n "$CORP_NAME" ] && ip -br addr show dev "$CORP_NAME" 2>/dev/null
|
||||
echo CHANGED
|
||||
exit 0
|
||||
|
||||
297
playbook/pxe-server-helpers/pxe-pick-nic
Executable file
297
playbook/pxe-server-helpers/pxe-pick-nic
Executable file
@@ -0,0 +1,297 @@
|
||||
#!/bin/bash
|
||||
# pxe-pick-nic - choose the imaging NIC by its cable, not by its MAC.
|
||||
#
|
||||
# Usage:
|
||||
# sudo pxe-pick-nic interactive: pull the imaging switch cable and
|
||||
# plug it back in; the port that blinked is it
|
||||
# pxe-pick-nic --auto for the first-boot playbook: the one wired
|
||||
# port with a link, if it is isolated
|
||||
# pxe-pick-nic --show print the saved choice
|
||||
#
|
||||
# The choice is saved in /etc/pxe-server/nic-pick.conf (imaging_mac=...).
|
||||
# The playbook uses it when site-vars.yml leaves pxe_nic_mac empty, and pins
|
||||
# that MAC from then on: the box never re-guesses at boot, so a moved cable
|
||||
# can never move 172.16.9.1 and DHCP onto another port.
|
||||
#
|
||||
# A port is accepted only when pxe-nic-check calls it "isolated": no GE
|
||||
# switch (LLDP/CDP), no 802.1X port authentication, and no other DHCP
|
||||
# server answering one DHCPDISCOVER. It listens CHECK_WINDOW seconds
|
||||
# (default 65: a Cisco switch sends CDP every 60 s). --auto also refuses a
|
||||
# port where more than ARP_MAX (default 10) hosts were heard: an imaging
|
||||
# switch at install time is nearly silent, a GE access VLAN is not, and
|
||||
# that holds even when the switch sends no LLDP/CDP and the probe is lost
|
||||
# (STP still listening). Anything else is refused and nothing is saved.
|
||||
# The check uses the site's corp_domain_suffixes and allowed_switches
|
||||
# (PXE_PICK_SUFFIXES / PXE_PICK_ALLOWED from the playbook, else
|
||||
# /etc/pxe-server/nic.conf), so a managed imaging switch can be allowed.
|
||||
#
|
||||
# Install rule for sites: during the install plug in ONLY the imaging
|
||||
# switch. Plug the GE cable into any other port afterwards; the catch-all
|
||||
# netplan entry makes it a DHCP client and dnsmasq never serves it.
|
||||
#
|
||||
# Exit: 0 chosen (or --show found one), 1 refused / could not choose,
|
||||
# 2 usage.
|
||||
|
||||
set -u
|
||||
|
||||
CONF=${PXE_PICK_CONF:-/etc/pxe-server/nic-pick.conf}
|
||||
HINT=${PXE_PICK_HINT:-/var/lib/pxe-firstboot/nic-hint}
|
||||
SYSFS_NET=${SYSFS_NET:-/sys/class/net}
|
||||
NIC_CHECK=${PXE_NIC_CHECK:-/usr/local/sbin/pxe-nic-check}
|
||||
CHECK_WINDOW=${CHECK_WINDOW:-65}
|
||||
ARP_MAX=${PXE_PICK_ARP_MAX:-10}
|
||||
NIC_CONF=${PXE_NIC_CONF:-/etc/pxe-server/nic.conf}
|
||||
SETTLE=${PXE_PICK_SETTLE:-8}
|
||||
WAIT=${PXE_PICK_WAIT:-180}
|
||||
|
||||
log() { echo "pxe-pick-nic: $*"; }
|
||||
|
||||
# Physical wired Ethernet ports: a device behind them, ARP type 1, not
|
||||
# wireless, not a bridge.
|
||||
wired() {
|
||||
local d n
|
||||
for d in "$SYSFS_NET"/*; do
|
||||
n=${d##*/}
|
||||
[ -e "$d/device" ] || continue
|
||||
[ "$(cat "$d/type" 2>/dev/null)" = 1 ] || continue
|
||||
[ -e "$d/wireless" ] || [ -e "$d/phy80211" ] && continue
|
||||
[ -e "$d/bridge" ] && continue
|
||||
echo "$n"
|
||||
done
|
||||
}
|
||||
|
||||
mac_of() { cat "$SYSFS_NET/$1/address" 2>/dev/null; }
|
||||
|
||||
# 1 = link, 0 = none. carrier is unreadable while the port is admin-down.
|
||||
carrier() { [ "$(cat "$SYSFS_NET/$1/carrier" 2>/dev/null)" = 1 ] && echo 1 || echo 0; }
|
||||
|
||||
# Bring every wired port up (no address is added) so its link can be seen.
|
||||
links_up() {
|
||||
local n up=0
|
||||
for n in $(wired); do
|
||||
if ! ip link show dev "$n" 2>/dev/null | grep -q '[<,]UP[,>]'; then
|
||||
ip link set dev "$n" up 2>/dev/null && up=1
|
||||
fi
|
||||
done
|
||||
[ "$up" -eq 1 ] && sleep "$SETTLE"
|
||||
return 0
|
||||
}
|
||||
|
||||
table() {
|
||||
local n
|
||||
for n in $(wired); do
|
||||
printf ' %-16s %s link %s\n' "$n" "$(mac_of "$n")" \
|
||||
"$([ "$(carrier "$n")" = 1 ] && echo up || echo down)"
|
||||
done
|
||||
}
|
||||
|
||||
# A pxe-nic-check config with only the site settings that matter here
|
||||
# (never the MAC keys: the port under test is given with --iface).
|
||||
check_config() {
|
||||
local f=$1 suffixes=${PXE_PICK_SUFFIXES:-} allowed=${PXE_PICK_ALLOWED:-}
|
||||
if [ -z "$suffixes$allowed" ] && [ -f "$NIC_CONF" ]; then
|
||||
suffixes=$(sed -n 's/^[[:space:]]*corp_domain_suffixes[[:space:]]*=[[:space:]]*//p' "$NIC_CONF" | tail -1)
|
||||
allowed=$(sed -n 's/^[[:space:]]*allowed_switches[[:space:]]*=[[:space:]]*//p' "$NIC_CONF" | tail -1)
|
||||
fi
|
||||
{
|
||||
echo "[nic]"
|
||||
[ -n "$suffixes" ] && echo "corp_domain_suffixes = $suffixes"
|
||||
[ -n "$allowed" ] && echo "allowed_switches = $allowed"
|
||||
} >"$f"
|
||||
}
|
||||
|
||||
# check IFACE [strict]: prints the verdict, returns 0 only for "isolated"
|
||||
# (with strict, also only when at most ARP_MAX hosts were heard).
|
||||
check() {
|
||||
local out verdict arp why cfg
|
||||
if [ ! -f "$NIC_CHECK" ]; then
|
||||
log "ERROR: $NIC_CHECK not found"
|
||||
return 1
|
||||
fi
|
||||
cfg=$(mktemp)
|
||||
check_config "$cfg"
|
||||
log "checking $1 is isolated: listening ${CHECK_WINDOW}s for GE switches and DHCP servers..."
|
||||
out=$(python3 "$NIC_CHECK" --iface "$1" --probe --window "$CHECK_WINDOW" --json --config "$cfg" 2>/dev/null)
|
||||
rm -f "$cfg"
|
||||
read -r verdict arp why < <(python3 -c '
|
||||
import json, sys
|
||||
try:
|
||||
e = json.load(sys.stdin)["interfaces"][0]
|
||||
ev = "; ".join(x for x in e["evidence"] if not x.endswith("ARP hosts seen"))
|
||||
print(e["verdict"], int(e.get("arp_hosts") or 0), ev or "-")
|
||||
except Exception as ex:
|
||||
print("error", 0, "pxe-nic-check gave no usable result: %s" % ex)
|
||||
' <<<"$out")
|
||||
if [ "$verdict" = isolated ]; then
|
||||
if [ "$arp" -gt "$ARP_MAX" ]; then
|
||||
log "$1: isolated by its switch and DHCP, but $arp hosts were heard (more than $ARP_MAX)"
|
||||
if [ "${2:-}" = strict ]; then
|
||||
log "REFUSED: a busy LAN is not an imaging switch at install time; it may be a GE port"
|
||||
log "with LLDP/CDP off. Pick it by its cable (sudo pxe-pick-nic) or set pxe_nic_mac."
|
||||
return 1
|
||||
fi
|
||||
log "WARNING: make sure this is the imaging switch, not a GE port with LLDP/CDP turned off."
|
||||
else
|
||||
log "$1: isolated (no GE switch, no 802.1X, no other DHCP server, $arp hosts heard)"
|
||||
fi
|
||||
return 0
|
||||
fi
|
||||
log "$1: verdict $verdict: $why"
|
||||
case "$verdict" in
|
||||
ge) log "That port is on the GE network (or a managed switch not in allowed_switches). It must never be the imaging port." ;;
|
||||
foreign-dhcp) log "Another DHCP server answers on that port (a second PXE server, or a routed LAN)." ;;
|
||||
no-link) log "That port lost its link during the check." ;;
|
||||
esac
|
||||
return 1
|
||||
}
|
||||
|
||||
save() {
|
||||
local n=$1 mac
|
||||
mac=$(mac_of "$n")
|
||||
mkdir -p "$(dirname "$CONF")"
|
||||
{
|
||||
echo "# Written by pxe-pick-nic $(date -Is). The imaging NIC; the playbook"
|
||||
echo "# pins it when site-vars.yml leaves pxe_nic_mac empty."
|
||||
echo "imaging_mac=$mac"
|
||||
echo "picked_iface=$n"
|
||||
} >"$CONF.tmp" && mv "$CONF.tmp" "$CONF"
|
||||
rm -f "$HINT"
|
||||
log "saved: imaging NIC = $n ($mac) in $CONF"
|
||||
}
|
||||
|
||||
hint() {
|
||||
mkdir -p "$(dirname "$HINT")" 2>/dev/null
|
||||
{ printf '%s\n' "$*" >"$HINT"; } 2>/dev/null
|
||||
}
|
||||
|
||||
saved_mac() {
|
||||
sed -n 's/^imaging_mac=\(\([0-9a-f]\{2\}:\)\{5\}[0-9a-f]\{2\}\)$/\1/p' "$CONF" 2>/dev/null | tail -1
|
||||
}
|
||||
|
||||
# --- modes ---------------------------------------------------------------------
|
||||
|
||||
mode_show() {
|
||||
local mac
|
||||
mac=$(saved_mac)
|
||||
[ -n "$mac" ] || { log "no saved choice in $CONF"; return 1; }
|
||||
echo "$mac"
|
||||
}
|
||||
|
||||
mode_auto() {
|
||||
local mac n with=() names
|
||||
mac=$(saved_mac)
|
||||
if [ -n "$mac" ]; then
|
||||
for n in $(wired); do
|
||||
if [ "$(mac_of "$n")" = "$mac" ]; then
|
||||
log "using the saved choice: $n ($mac)"
|
||||
echo "$mac"
|
||||
return 0
|
||||
fi
|
||||
done
|
||||
log "the saved imaging NIC $mac is not on this box any more"
|
||||
hint "The saved imaging NIC $mac is gone (dongle replaced?). Log in and run: sudo pxe-pick-nic"
|
||||
return 1
|
||||
fi
|
||||
links_up
|
||||
for n in $(wired); do
|
||||
[ "$(carrier "$n")" = 1 ] && with+=("$n")
|
||||
done
|
||||
log "wired ports:"
|
||||
table
|
||||
if [ ${#with[@]} -eq 0 ]; then
|
||||
log "REFUSED: no wired port has a link. Plug the imaging switch into the port to use."
|
||||
hint "Could not pick the imaging port: no port has a link. Plug in the imaging switch, then log in and run: sudo pxe-pick-nic"
|
||||
return 1
|
||||
fi
|
||||
if [ ${#with[@]} -gt 1 ]; then
|
||||
names="${with[*]}"
|
||||
log "REFUSED: ${#with[@]} ports have a link ($names). During the install only the imaging switch may be plugged in."
|
||||
hint "Could not pick the imaging port: ${#with[@]} ports have a link ($names). Log in and run: sudo pxe-pick-nic"
|
||||
return 1
|
||||
fi
|
||||
n=${with[0]}
|
||||
if ! check "$n" strict; then
|
||||
hint "Could not pick the imaging port: $n is not isolated (see /var/log/first-boot.log). Plug only the imaging switch in, then log in and run: sudo pxe-pick-nic"
|
||||
return 1
|
||||
fi
|
||||
save "$n" >&2
|
||||
mac_of "$n"
|
||||
}
|
||||
|
||||
# Wait until exactly one port changes its link; print "NAME NEWSTATE".
|
||||
wait_change() {
|
||||
local -A before=()
|
||||
local n t=0 changed=()
|
||||
for n in $(wired); do before[$n]=$(carrier "$n"); done
|
||||
while [ "$t" -lt $((WAIT * 2)) ]; do
|
||||
sleep 0.5
|
||||
t=$((t + 1))
|
||||
changed=()
|
||||
for n in "${!before[@]}"; do
|
||||
[ "$(carrier "$n")" != "${before[$n]}" ] && changed+=("$n")
|
||||
done
|
||||
if [ ${#changed[@]} -eq 1 ]; then
|
||||
sleep 1 # a flapping connector settles
|
||||
n=${changed[0]}
|
||||
echo "$n $(carrier "$n")"
|
||||
return 0
|
||||
fi
|
||||
if [ ${#changed[@]} -gt 1 ]; then
|
||||
log "more than one port changed (${changed[*]}); move one cable at a time" >&2
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
log "nothing changed in ${WAIT}s" >&2
|
||||
return 1
|
||||
}
|
||||
|
||||
mode_interactive() {
|
||||
local n state again ans
|
||||
[ -t 0 ] || { log "interactive mode needs a terminal (use --auto from scripts)"; return 2; }
|
||||
[ "$(id -u)" -eq 0 ] || { log "run it as root: sudo pxe-pick-nic"; return 2; }
|
||||
links_up
|
||||
echo
|
||||
echo "Pick the imaging NIC: the port cabled to the isolated imaging switch."
|
||||
echo "Wired ports now:"
|
||||
table
|
||||
echo
|
||||
echo "Pull the imaging switch cable out of this server now"
|
||||
echo "(or, if it is not plugged in yet, plug it in now). Waiting up to ${WAIT}s..."
|
||||
read -r n state < <(wait_change) || return 1
|
||||
[ -n "${n:-}" ] || return 1
|
||||
if [ "$state" = 0 ]; then
|
||||
echo "$n lost its link. Plug the cable back into the SAME port..."
|
||||
read -r again state < <(wait_change) || return 1
|
||||
if [ "${again:-}" != "$n" ] || [ "$state" != 1 ]; then
|
||||
log "expected $n to come back up, saw ${again:-nothing} $state; try again"
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
sleep "$SETTLE"
|
||||
echo "Imaging port: $n ($(mac_of "$n"))"
|
||||
check "$n" || { log "nothing saved"; return 1; }
|
||||
read -rp "Use $n ($(mac_of "$n")) as the imaging NIC? [y/N] " ans
|
||||
[[ "$ans" =~ ^[Yy]$ ]] || { log "nothing saved"; return 1; }
|
||||
save "$n"
|
||||
if [ -e /var/lib/pxe-firstboot.failed ] && [ -x /opt/first-boot.sh ]; then
|
||||
read -rp "Run the server setup again now? [Y/n] " ans
|
||||
if [[ ! "$ans" =~ ^[Nn]$ ]]; then
|
||||
rm -f /var/lib/pxe-firstboot/attempts
|
||||
/opt/first-boot.sh 2>&1 | tee -a /var/log/first-boot.log
|
||||
return "${PIPESTATUS[0]}"
|
||||
fi
|
||||
echo "Run it later with: sudo rm -f /var/lib/pxe-firstboot/attempts; sudo /opt/first-boot.sh"
|
||||
else
|
||||
echo "Saved. If the server is already set up, re-run the playbook from the"
|
||||
echo "installer USB to move the imaging config onto this port."
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
case "${1:-}" in
|
||||
--auto) mode_auto ;;
|
||||
--show) mode_show ;;
|
||||
"") mode_interactive ;;
|
||||
-h|--help) sed -n '2,/^$/p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
|
||||
*) echo "usage: $0 [--auto | --show]" >&2; exit 2 ;;
|
||||
esac
|
||||
@@ -73,19 +73,30 @@
|
||||
# image roots (on a rebuild the _media views may not exist yet).
|
||||
zscaler_ca_share_glob: "{{ samba_share }}/_media/*/Deploy/Applications/ZscalerCommercialCertificate-2048-SHA256.crt {{ samba_share }}/gea-*/Deploy/Applications/ZscalerCommercialCertificate-2048-SHA256.crt"
|
||||
# --- The two NICs (site standard) ---
|
||||
# Every server has two NICs: the imaging NIC (isolated imaging LAN,
|
||||
# static pxe_server_ip, the ONLY NIC that serves DHCP/TFTP/HTTP boot/SMB)
|
||||
# and the GE corporate NIC (DHCP client; SSH and the webapp on 9009 for
|
||||
# admins, outbound updates through Zscaler). Both MACs are REQUIRED:
|
||||
# set them in playbook/site-vars.yml on the USB (copy
|
||||
# site-vars.example.yml) or with -e. The playbook stops before it
|
||||
# touches anything if either is unset or not on this box. Read them
|
||||
# with: ip -br link
|
||||
# The imaging NIC (isolated imaging LAN, static pxe_server_ip, the ONLY
|
||||
# NIC that serves DHCP/TFTP/HTTP boot/SMB) is pinned by MAC. Leave
|
||||
# pxe_nic_mac empty and the first run picks it (pxe-pick-nic --auto):
|
||||
# the one wired port with a link, accepted only if pxe-nic-check calls
|
||||
# it isolated, saved in /etc/pxe-server/nic-pick.conf and pinned from
|
||||
# then on. So during the install plug in ONLY the imaging switch. If it
|
||||
# cannot tell, it stops and the console says to run sudo pxe-pick-nic
|
||||
# (pull the imaging cable and plug it back in).
|
||||
#
|
||||
# The GE corporate NIC (DHCP client; SSH and the webapp on 9009) is
|
||||
# optional:
|
||||
# corp_nic_mac empty (default): every other wired port (en*, eth*) is
|
||||
# a DHCP client, so GE can be plugged into any of them, then or
|
||||
# later. dnsmasq never serves them (interface= pins it to the
|
||||
# imaging NIC) and they ignore our own DHCP offers.
|
||||
# corp_nic_mac set: that one port is pinned and named corp_nic_name,
|
||||
# as on the WJ server; other ports stay unconfigured.
|
||||
pxe_nic_mac: ""
|
||||
corp_nic_mac: ""
|
||||
# Stable names, set by netplan (match macaddress + set-name). dnsmasq,
|
||||
# UFW and Samba refer to these. A site may keep its existing names
|
||||
# (e.g. pxe_nic_name: enx34c8d6b11010, corp_nic_name: enp0s31f6).
|
||||
# UFW and Samba refer to these. With corp_nic_mac empty they must not
|
||||
# look like en*/eth* (the catch-all would match them by name). With both
|
||||
# MACs pinned a site may keep its existing names (WJ: pxe_nic_name:
|
||||
# enx34c8d6b11010, corp_nic_name: enp0s31f6).
|
||||
pxe_nic_name: "pxe0"
|
||||
corp_nic_name: "corp0"
|
||||
# Imaging LAN addressing. NOTE: 172.16.9.1 is also hard-coded in the
|
||||
@@ -101,6 +112,7 @@
|
||||
corp_route_metric: 100
|
||||
# NIC guard (pxe-nic-check): dnsmasq must never serve DHCP/PXE on the GE
|
||||
# corporate network. These are derived from pxe_nic_mac / corp_nic_mac
|
||||
# (pxe_ge_mac empty: the guard takes the NIC holding a DHCP lease)
|
||||
# (one source of truth) and written to /etc/pxe-server/nic.conf; do not
|
||||
# set them separately (the playbook fails if they disagree). A MAC of a
|
||||
# bridge port resolves to the bridge.
|
||||
@@ -210,29 +222,77 @@
|
||||
- Tools
|
||||
|
||||
tasks:
|
||||
# --- The two NICs: validate, then pin by MAC (netplan) ---
|
||||
# --- The NICs: pick / validate, then pin by MAC (netplan) ---
|
||||
# Replaces the old guess (first en* without 172.16.9.1 / the default
|
||||
# route), which on a two-NIC box could pick the GE NIC.
|
||||
# route), which on a two-NIC box could pick the GE NIC. The pick runs
|
||||
# from the USB copies, before anything on the box changes, and only
|
||||
# ever at install time: once pinned, nothing re-guesses at boot.
|
||||
# Installed first (they change no network config): when the pick
|
||||
# refuses, the play stops here and the console tells the operator to
|
||||
# run sudo pxe-pick-nic, which needs both on the box (first-boot.sh
|
||||
# unmounts the USB).
|
||||
- name: "NICs: install pxe-nic-check and pxe-pick-nic before the pick"
|
||||
copy:
|
||||
src: "{{ usb_mount }}/pxe-server-helpers/{{ item }}"
|
||||
dest: "/usr/local/sbin/{{ item }}"
|
||||
mode: '0755'
|
||||
loop: [pxe-nic-check, pxe-pick-nic]
|
||||
|
||||
- name: "NICs: pick the imaging NIC (pxe_nic_mac is not set in site-vars)"
|
||||
command: /usr/local/sbin/pxe-pick-nic --auto
|
||||
environment:
|
||||
PXE_PICK_SUFFIXES: "{{ pxe_corp_domain_suffixes }}"
|
||||
PXE_PICK_ALLOWED: "{{ pxe_nic_allowed_switches }}"
|
||||
register: nic_pick
|
||||
changed_when: false
|
||||
failed_when: false
|
||||
when: pxe_nic_mac | length == 0
|
||||
|
||||
- name: "NICs: pick result"
|
||||
debug:
|
||||
msg: "{{ nic_pick.stdout_lines }}"
|
||||
when: pxe_nic_mac | length == 0
|
||||
|
||||
- name: "NICs: the imaging NIC could not be picked"
|
||||
fail:
|
||||
msg: >-
|
||||
Could not pick the imaging NIC, nothing was changed:
|
||||
{{ nic_pick.stdout_lines | select('search', 'REFUSED|verdict|gone|ERROR') | join(' ') }}
|
||||
During the install plug in ONLY the imaging switch, or log in and
|
||||
run: sudo pxe-pick-nic (pull the imaging cable and plug it back in).
|
||||
Or set pxe_nic_mac in playbook/site-vars.yml.
|
||||
when:
|
||||
- pxe_nic_mac | length == 0
|
||||
- nic_pick.rc != 0 or not (nic_pick.stdout_lines | last | default('') is match('^([0-9a-f]{2}:){5}[0-9a-f]{2}$'))
|
||||
|
||||
- name: "NICs: pin the picked imaging NIC"
|
||||
set_fact:
|
||||
pxe_nic_mac: "{{ nic_pick.stdout_lines | last }}"
|
||||
when: pxe_nic_mac | length == 0
|
||||
|
||||
- name: "NICs: check the per-site NIC vars"
|
||||
assert:
|
||||
that:
|
||||
- pxe_nic_mac | length > 0
|
||||
- corp_nic_mac | length > 0
|
||||
- pxe_nic_mac is match('^([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}$')
|
||||
- corp_nic_mac is match('^([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}$')
|
||||
- corp_nic_mac | length == 0 or corp_nic_mac is match('^([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}$')
|
||||
- pxe_nic_mac | lower != corp_nic_mac | lower
|
||||
- pxe_imaging_mac | lower == pxe_nic_mac | lower
|
||||
- pxe_ge_mac | lower == corp_nic_mac | lower
|
||||
- pxe_nic_name | length > 0 and pxe_nic_name | length <= 15
|
||||
- corp_nic_name | length > 0 and corp_nic_name | length <= 15
|
||||
- pxe_nic_name != corp_nic_name
|
||||
- corp_nic_mac | length > 0 or not (pxe_nic_name is match('^(en|eth)'))
|
||||
fail_msg: >-
|
||||
Set pxe_nic_mac (imaging NIC) and corp_nic_mac (GE corporate NIC)
|
||||
to two different MACs (aa:bb:cc:dd:ee:ff) in playbook/site-vars.yml
|
||||
(copy site-vars.example.yml) or with -e, then re-run. Do not set
|
||||
pxe_imaging_mac / pxe_ge_mac yourself; they follow these. Names
|
||||
(pxe_nic_name, corp_nic_name) must differ and be 1-15 chars.
|
||||
Now: pxe_nic_mac='{{ pxe_nic_mac }}' corp_nic_mac='{{ corp_nic_mac }}'.
|
||||
pxe_nic_mac (imaging NIC) must be a MAC (aa:bb:cc:dd:ee:ff);
|
||||
corp_nic_mac (GE NIC) empty or a different MAC. Set them in
|
||||
playbook/site-vars.yml (copy site-vars.example.yml) or with -e,
|
||||
then re-run. Do not set pxe_imaging_mac / pxe_ge_mac yourself;
|
||||
they follow these (an -e pxe_nic_mac="" overrides the pick too:
|
||||
drop it). Names (pxe_nic_name, corp_nic_name) must differ
|
||||
and be 1-15 chars, and with corp_nic_mac empty pxe_nic_name must
|
||||
not start with en or eth (the catch-all for the other ports
|
||||
matches those). Now: pxe_nic_mac='{{ pxe_nic_mac }}'
|
||||
corp_nic_mac='{{ corp_nic_mac }}' pxe_nic_name='{{ pxe_nic_name }}'.
|
||||
quiet: yes
|
||||
|
||||
- name: "NICs: list the physical NICs on this box"
|
||||
@@ -244,15 +304,15 @@
|
||||
register: nic_list
|
||||
changed_when: false
|
||||
|
||||
- name: "NICs: both MACs must be on this box"
|
||||
- name: "NICs: the pinned MACs must be on this box"
|
||||
vars:
|
||||
box_macs: "{{ nic_list.stdout_lines | map('regex_replace', '^.* ', '') | map('lower') | list }}"
|
||||
assert:
|
||||
that:
|
||||
- pxe_nic_mac | lower in box_macs
|
||||
- corp_nic_mac | lower in box_macs
|
||||
- corp_nic_mac | length == 0 or corp_nic_mac | lower in box_macs
|
||||
fail_msg: >-
|
||||
{{ 'pxe_nic_mac ' ~ pxe_nic_mac ~ ' is not on this box. ' if pxe_nic_mac | lower not in box_macs else '' }}{{ 'corp_nic_mac ' ~ corp_nic_mac ~ ' is not on this box. ' if corp_nic_mac | lower not in box_macs else '' }}Nothing was changed (the network config would have left that NIC
|
||||
{{ 'pxe_nic_mac ' ~ pxe_nic_mac ~ ' is not on this box. ' if pxe_nic_mac | lower not in box_macs else '' }}{{ 'corp_nic_mac ' ~ corp_nic_mac ~ ' is not on this box. ' if corp_nic_mac and corp_nic_mac | lower not in box_macs else '' }}Nothing was changed (the network config would have left that NIC
|
||||
unconfigured). Physical NICs here (name MAC):
|
||||
{{ nic_list.stdout_lines | join('; ') }}.
|
||||
Fix playbook/site-vars.yml and re-run.
|
||||
@@ -396,7 +456,7 @@
|
||||
bash {{ usb_mount }}/pxe-server-helpers/pxe-netplan-apply.sh --preflight
|
||||
/run/pxe-server-netplan.check.yaml
|
||||
{{ pxe_nic_name }} {{ pxe_nic_mac | lower }}
|
||||
{{ corp_nic_name }} {{ corp_nic_mac | lower }}
|
||||
{{ (corp_nic_name ~ ' ' ~ corp_nic_mac | lower) if corp_nic_mac else '' }}
|
||||
environment:
|
||||
PXE_SUBNET: "{{ pxe_subnet }}"
|
||||
PXE_SSH_CLIENT: "{{ pxe_ssh_client }}"
|
||||
@@ -439,11 +499,57 @@
|
||||
dest: /etc/pxe-server-netplan.yaml
|
||||
mode: '0600'
|
||||
|
||||
# The catch-all ports (corp_nic_mac empty) are DHCP clients on whatever
|
||||
# they are cabled to. Cabled to the imaging switch by mistake, they would
|
||||
# take an address from our own dnsmasq and put a second interface in the
|
||||
# imaging subnet; DenyList makes them ignore offers from pxe_server_ip.
|
||||
- name: "NICs: drop-in folders for the catch-all ports"
|
||||
file:
|
||||
path: "/etc/systemd/network/10-netplan-{{ item }}.network.d"
|
||||
state: directory
|
||||
mode: '0755'
|
||||
loop: [zz-en, zz-eth]
|
||||
when: corp_nic_mac | length == 0
|
||||
|
||||
- name: "NICs: catch-all ports ignore our own DHCP offers"
|
||||
copy:
|
||||
dest: "/etc/systemd/network/10-netplan-{{ item }}.network.d/90-pxe-deny-own-dhcp.conf"
|
||||
mode: '0644'
|
||||
content: |
|
||||
# Managed by pxe_server_setup.yml. Never take a lease from this
|
||||
# server's own dnsmasq (an imaging switch on a non-imaging port).
|
||||
[DHCPv4]
|
||||
DenyList={{ pxe_server_ip }}
|
||||
loop: [zz-en, zz-eth]
|
||||
when: corp_nic_mac | length == 0
|
||||
|
||||
- name: "NICs: no catch-all drop-ins with a pinned GE NIC"
|
||||
file:
|
||||
path: "/etc/systemd/network/10-netplan-{{ item }}.network.d"
|
||||
state: absent
|
||||
loop: [zz-en, zz-eth]
|
||||
when: corp_nic_mac | length > 0
|
||||
|
||||
- name: "NICs: create /etc/pxe-server"
|
||||
file:
|
||||
path: /etc/pxe-server
|
||||
state: directory
|
||||
mode: '0755'
|
||||
|
||||
- name: "NICs: record the pinned imaging NIC for pxe-pick-nic --show"
|
||||
copy:
|
||||
dest: /etc/pxe-server/nic-pick.conf
|
||||
mode: '0644'
|
||||
content: |
|
||||
# Written by pxe_server_setup.yml. The imaging NIC the playbook pins.
|
||||
imaging_mac={{ pxe_nic_mac | lower }}
|
||||
when: nic_pick is not skipped and nic_pick.rc == 0
|
||||
|
||||
- name: "NICs: validate, install and apply netplan (rolls back on failure)"
|
||||
command: >-
|
||||
/usr/local/sbin/pxe-netplan-apply.sh /etc/pxe-server-netplan.yaml
|
||||
{{ pxe_nic_name }} {{ pxe_nic_mac | lower }}
|
||||
{{ corp_nic_name }} {{ corp_nic_mac | lower }}
|
||||
{{ (corp_nic_name ~ ' ' ~ corp_nic_mac | lower) if corp_nic_mac else '' }}
|
||||
environment:
|
||||
PXE_SUBNET: "{{ pxe_subnet }}"
|
||||
PXE_SSH_CLIENT: "{{ pxe_ssh_client }}"
|
||||
@@ -520,7 +626,7 @@
|
||||
listen-address={{ pxe_server_ip }}
|
||||
bind-dynamic
|
||||
# Interface pinning (serve {{ pxe_nic_name }} only, never
|
||||
# {{ corp_nic_name }}) is in /etc/dnsmasq.d/pxe-nic-only.conf.
|
||||
# {{ corp_nic_name if corp_nic_mac else 'the other ports' }}) is in /etc/dnsmasq.d/pxe-nic-only.conf.
|
||||
dhcp-range={{ pxe_dhcp_start }},{{ pxe_dhcp_end }},{{ pxe_dhcp_lease }}
|
||||
# No default gateway (option 3) and no DNS (option 6) handed out:
|
||||
# the PXE network is isolated and the PXE server does not forward
|
||||
@@ -576,6 +682,37 @@
|
||||
mode: '0644'
|
||||
backup: yes
|
||||
|
||||
# The second layer under interface=: dnsmasq offers only on an interface
|
||||
# with an address inside a dhcp-range. With every range in the imaging
|
||||
# subnet, a GE port (10.x) matches none even if the pin were lost. A
|
||||
# proxy range or one outside the subnet would remove that layer.
|
||||
- name: "dnsmasq: every dhcp-range is in the imaging subnet, none is proxy"
|
||||
shell: |
|
||||
set -u
|
||||
bad=0
|
||||
# Files the dnsmasq conf-dir skips (dot files, ~ backups, .dpkg-*,
|
||||
# .bak) are skipped here too, so a backup never refuses the run.
|
||||
FILES=(/etc/dnsmasq.conf)
|
||||
for f in /etc/dnsmasq.d/*; do
|
||||
b=${f##*/}
|
||||
case "$b" in .*|*~|*.dpkg-*|*.bak|*.bak.*) continue ;; esac
|
||||
[ -f "$f" ] && FILES+=("$f")
|
||||
done
|
||||
while IFS= read -r line; do
|
||||
range=${line#*dhcp-range=}
|
||||
first=$(printf '%s' "$range" | tr ',' '\n' | grep -Em1 '^[0-9]+(\.[0-9]+){3}$')
|
||||
if printf '%s' "$range" | grep -qi proxy; then
|
||||
echo "REFUSED proxy DHCP: $line" >&2; bad=1
|
||||
elif [ -z "$first" ] || ! python3 -c 'import ipaddress,sys; sys.exit(ipaddress.ip_address(sys.argv[1]) not in ipaddress.ip_network(sys.argv[2]))' "$first" "{{ pxe_subnet }}"; then
|
||||
echo "REFUSED dhcp-range outside {{ pxe_subnet }}: $line" >&2; bad=1
|
||||
fi
|
||||
done < <(grep -Hs '^[[:space:]]*dhcp-range=' "${FILES[@]}" | sed 's/[[:space:]]*#.*//')
|
||||
[ "$bad" -eq 0 ] || exit 1
|
||||
echo "dhcp-range ok"
|
||||
args:
|
||||
executable: /bin/bash
|
||||
changed_when: false
|
||||
|
||||
# --- NIC guard: keep dnsmasq off the GE corporate network ---
|
||||
# pxe-nic-check classifies each NIC from what it hears (LLDP/CDP/802.1X
|
||||
# from managed switches, foreign DHCP offers and their domain). The
|
||||
@@ -1927,7 +2064,21 @@
|
||||
loop: "{{ ufw_corp_rules }}"
|
||||
loop_control:
|
||||
label: "{{ corp_nic_name }} {{ item.port }}/{{ item.proto }}"
|
||||
when: corp_admin_cidrs | length == 0
|
||||
when: corp_admin_cidrs | length == 0 and corp_nic_mac | length > 0
|
||||
|
||||
# No pinned GE NIC (corp_nic_mac empty): the GE port has no fixed name,
|
||||
# so SSH and the webapp are allowed by source, on any interface. The
|
||||
# imaging NIC already allows both, so this opens nothing new there.
|
||||
- name: "UFW: allow SSH and the webapp from all of GE (no pinned GE NIC)"
|
||||
ufw:
|
||||
rule: allow
|
||||
direction: in
|
||||
port: "{{ item.port }}"
|
||||
proto: "{{ item.proto }}"
|
||||
loop: "{{ ufw_corp_rules }}"
|
||||
loop_control:
|
||||
label: "any {{ item.port }}/{{ item.proto }}"
|
||||
when: corp_admin_cidrs | length == 0 and corp_nic_mac | length == 0
|
||||
|
||||
- name: "UFW: WARNING, SSH and the root webapp are open to all of GE"
|
||||
debug:
|
||||
@@ -1935,7 +2086,7 @@
|
||||
- "!!! WARNING: corp_admin_cidrs is empty. SSH (password auth, shared"
|
||||
- "!!! 'pxe' password) and the webapp on 9009 (no login, runs as root)"
|
||||
- "!!! accept connections from ANY host on the GE corporate network via"
|
||||
- "!!! {{ corp_nic_name }}. Set corp_admin_cidrs in playbook/site-vars.yml"
|
||||
- "!!! {{ corp_nic_name if corp_nic_mac else 'any GE port' }}. Set corp_admin_cidrs in playbook/site-vars.yml"
|
||||
- "!!! to the admin subnets (docs/SITE-CUSTOMIZATION.md) and re-run."
|
||||
when: corp_admin_cidrs | length == 0
|
||||
|
||||
@@ -1950,7 +2101,19 @@
|
||||
loop: "{{ ufw_corp_rules | product(corp_admin_cidrs) | list }}"
|
||||
loop_control:
|
||||
label: "{{ corp_nic_name }} from {{ item.1 }} {{ item.0.port }}/{{ item.0.proto }}"
|
||||
when: corp_admin_cidrs | length > 0
|
||||
when: corp_admin_cidrs | length > 0 and corp_nic_mac | length > 0
|
||||
|
||||
- name: "UFW: allow SSH and the webapp from corp_admin_cidrs only (no pinned GE NIC)"
|
||||
ufw:
|
||||
rule: allow
|
||||
direction: in
|
||||
from_ip: "{{ item.1 }}"
|
||||
port: "{{ item.0.port }}"
|
||||
proto: "{{ item.0.proto }}"
|
||||
loop: "{{ ufw_corp_rules | product(corp_admin_cidrs) | list }}"
|
||||
loop_control:
|
||||
label: "any from {{ item.1 }} {{ item.0.port }}/{{ item.0.proto }}"
|
||||
when: corp_admin_cidrs | length > 0 and corp_nic_mac | length == 0
|
||||
|
||||
- name: "UFW: drop the any-source rules on the GE corporate NIC (restricted now)"
|
||||
ufw:
|
||||
@@ -1963,7 +2126,7 @@
|
||||
loop: "{{ ufw_corp_rules }}"
|
||||
loop_control:
|
||||
label: "{{ corp_nic_name }} any {{ item.port }}/{{ item.proto }}"
|
||||
when: corp_admin_cidrs | length > 0
|
||||
when: corp_admin_cidrs | length > 0 and corp_nic_mac | length > 0
|
||||
|
||||
# Only drop the old any-interface rules (allow 22 among them) once both
|
||||
# named NICs exist on the right MACs and the per-interface SSH rules are
|
||||
@@ -1975,29 +2138,46 @@
|
||||
shell: |
|
||||
set -u
|
||||
bad=0
|
||||
for pair in "{{ pxe_nic_name }}={{ pxe_nic_mac | lower }}" "{{ corp_nic_name }}={{ corp_nic_mac | lower }}"; do
|
||||
PAIRS=("{{ pxe_nic_name }}={{ pxe_nic_mac | lower }}")
|
||||
NAMES=("{{ pxe_nic_name }}")
|
||||
{% if corp_nic_mac %}
|
||||
PAIRS+=("{{ corp_nic_name }}={{ corp_nic_mac | lower }}")
|
||||
NAMES+=("{{ corp_nic_name }}")
|
||||
{% endif %}
|
||||
for pair in "${PAIRS[@]}"; do
|
||||
n=${pair%%=*}; m=${pair#*=}
|
||||
if [ "$(cat "/sys/class/net/$n/address" 2>/dev/null)" != "$m" ]; then
|
||||
echo "ERROR: interface $n with MAC $m is not present" >&2; bad=1
|
||||
fi
|
||||
done
|
||||
added=$(ufw show added 2>/dev/null)
|
||||
for n in "{{ pxe_nic_name }}" "{{ corp_nic_name }}"; do
|
||||
for n in "${NAMES[@]}"; do
|
||||
if ! grep -E "allow in on $n( |$)" <<<"$added" | grep -qE 'port 22 proto tcp'; then
|
||||
echo "ERROR: no UFW 'allow in on $n ... port 22 proto tcp' rule" >&2; bad=1
|
||||
fi
|
||||
done
|
||||
{% if not corp_nic_mac %}
|
||||
# No pinned GE NIC: GE-side SSH is an any-interface rule, by source.
|
||||
if ! grep -E '^ufw allow' <<<"$added" | grep -v ' on ' | grep -qE '(port 22( |$)| 22/tcp( |$))'; then
|
||||
echo "ERROR: no any-interface UFW rule for port 22 (GE side)" >&2; bad=1
|
||||
fi
|
||||
{% endif %}
|
||||
[ "$bad" -eq 0 ] || { echo "the old any-interface rules are kept" >&2; exit 1; }
|
||||
echo "both NICs present, per-interface SSH rules in place"
|
||||
register: ufw_verify
|
||||
changed_when: false
|
||||
|
||||
# With no pinned GE NIC and corp_admin_cidrs empty, "allow 22" and
|
||||
# "allow 9009" on any interface ARE the GE-side rules: keep them.
|
||||
- name: "UFW: remove the old any-interface rules"
|
||||
ufw:
|
||||
rule: allow
|
||||
port: "{{ item.port }}"
|
||||
proto: "{{ item.proto }}"
|
||||
delete: yes
|
||||
when: >-
|
||||
corp_nic_mac | length > 0 or corp_admin_cidrs | length > 0
|
||||
or item.port not in (ufw_corp_rules | map(attribute='port') | list)
|
||||
loop:
|
||||
- { port: "22", proto: tcp }
|
||||
- { port: "67", proto: udp }
|
||||
|
||||
@@ -4,16 +4,22 @@
|
||||
# loads it on first boot; -e on the command line still overrides it.
|
||||
# site-vars.yml is git-ignored: each site's MACs stay with that site.
|
||||
#
|
||||
# REQUIRED: the two NICs, by MAC (ip -br link on the box, or the label).
|
||||
# imaging NIC = cable to the isolated imaging switch (serves DHCP/PXE/SMB)
|
||||
# corp NIC = cable to the GE corporate network (SSH + webapp 9009 only)
|
||||
# The playbook changes nothing on the network until both are set, differ,
|
||||
# and are both present on the box.
|
||||
# The NICs, by MAC (ip -br link on the box, or the label). Both optional:
|
||||
# pxe_nic_mac imaging NIC = cable to the isolated imaging switch (serves
|
||||
# DHCP/PXE/SMB). Empty: the first boot picks the one wired
|
||||
# port with a link, if pxe-nic-check calls it isolated, and
|
||||
# pins it. So during the install plug in ONLY the imaging
|
||||
# switch. If it cannot tell, the console says to log in and
|
||||
# run: sudo pxe-pick-nic (pull the imaging cable, plug it in).
|
||||
# corp_nic_mac GE corporate NIC (SSH + webapp 9009). Empty: every other
|
||||
# wired port is a DHCP client, so GE can be plugged into any
|
||||
# of them, now or later. Set: only that port, named corp0.
|
||||
pxe_nic_mac: "" # e.g. "34:c8:d6:b1:10:10"
|
||||
corp_nic_mac: "" # e.g. "20:88:10:af:0d:e5"
|
||||
|
||||
# Optional: stable interface names (default pxe0 / corp0). Keep a box's
|
||||
# existing names instead, e.g. on the WJ server:
|
||||
# Optional: stable interface names (default pxe0 / corp0). With corp_nic_mac
|
||||
# empty, pxe_nic_name must not start with en or eth. With both MACs set a
|
||||
# box may keep its existing names, e.g. on the WJ server:
|
||||
# pxe_nic_name: enx34c8d6b11010
|
||||
# corp_nic_name: enp0s31f6
|
||||
|
||||
|
||||
@@ -4,5 +4,11 @@
|
||||
# port=0 / listen-address / bind-dynamic in /etc/dnsmasq.conf and the
|
||||
# pxe-nic-check guard (docs/pxe-nic-guard.md).
|
||||
interface={{ pxe_nic_name }}
|
||||
{% if corp_nic_mac %}
|
||||
except-interface={{ corp_nic_name }}
|
||||
no-dhcp-interface={{ corp_nic_name }}
|
||||
{% else %}
|
||||
# No pinned GE NIC (corp_nic_mac empty): every other wired port is a DHCP
|
||||
# client with no fixed name. interface= alone keeps dnsmasq off them, and
|
||||
# its only dhcp-range is the imaging subnet, so a GE address matches none.
|
||||
{% endif %}
|
||||
|
||||
@@ -3,6 +3,7 @@
|
||||
# replaced on the next playbook run; the previous file is kept under
|
||||
# /var/backups/netplan-<timestamp>/.
|
||||
#
|
||||
{% if corp_nic_mac %}
|
||||
# Two NICs, each matched by MAC and given a stable name:
|
||||
# pxe-lan imaging LAN, static, no gateway, no DNS. The ONLY NIC that
|
||||
# serves DHCP/TFTP/HTTP boot/SMB. ignore-carrier: the address
|
||||
@@ -11,6 +12,24 @@
|
||||
# Apache without {{ pxe_server_ip }} (networkd
|
||||
# ConfigureWithoutCarrier= + IgnoreCarrierLoss=).
|
||||
# ge-corp GE corporate network, DHCP client (SSH + webapp 9009, updates).
|
||||
{% else %}
|
||||
# The imaging NIC pinned by MAC, every other wired port a DHCP client:
|
||||
# pxe-lan imaging LAN, matched by MAC and given a stable name. Static,
|
||||
# no gateway, no DNS. The ONLY NIC that serves DHCP/TFTP/HTTP
|
||||
# boot/SMB. ignore-carrier: the address is put on (and kept)
|
||||
# without link, so a switch that is off or a cable pulled at
|
||||
# boot does not leave dnsmasq, Samba and Apache without
|
||||
# {{ pxe_server_ip }} (networkd ConfigureWithoutCarrier= +
|
||||
# IgnoreCarrierLoss=).
|
||||
# zz-en, zz-eth every other wired port: DHCP client, so the GE cable
|
||||
# can go into any of them, at install or later. No set-name.
|
||||
# networkd applies the first matching .network file in name
|
||||
# order: 10-netplan-pxe-lan sorts before 10-netplan-zz-*, and
|
||||
# {{ pxe_nic_name }} does not look like en*/eth* (the playbook
|
||||
# checks), so the imaging NIC never falls into these. dnsmasq
|
||||
# never serves them (interface= pins it to {{ pxe_nic_name }}),
|
||||
# and a drop-in makes them ignore offers from {{ pxe_server_ip }}.
|
||||
{% endif %}
|
||||
network:
|
||||
version: 2
|
||||
renderer: networkd
|
||||
@@ -25,6 +44,7 @@ network:
|
||||
optional: true
|
||||
addresses:
|
||||
- {{ pxe_server_ip }}/{{ pxe_prefix }}
|
||||
{% if corp_nic_mac %}
|
||||
ge-corp:
|
||||
match:
|
||||
macaddress: "{{ corp_nic_mac | lower }}"
|
||||
@@ -32,3 +52,15 @@ network:
|
||||
dhcp4: true
|
||||
dhcp4-overrides:
|
||||
route-metric: {{ corp_route_metric }}
|
||||
{% else %}
|
||||
{% for id, glob in [('zz-en', 'en*'), ('zz-eth', 'eth*')] %}
|
||||
{{ id }}:
|
||||
match:
|
||||
name: "{{ glob }}"
|
||||
dhcp4: true
|
||||
dhcp6: false
|
||||
optional: true
|
||||
dhcp4-overrides:
|
||||
route-metric: {{ corp_route_metric }}
|
||||
{% endfor %}
|
||||
{% endif %}
|
||||
|
||||
@@ -34,12 +34,12 @@
|
||||
# Engineering Layer...: scripts/capture-site-content.sh).
|
||||
# Without it a new server cannot build standard or
|
||||
# engineer until that content is copied over by hand.
|
||||
# --site-vars FILE put FILE on CIDATA as playbook/site-vars.yml (the
|
||||
# per-site NIC MACs; see playbook/site-vars.example.yml)
|
||||
# --no-site-vars build without site vars (the first-boot playbook then
|
||||
# stops until they are set on the server). One of the
|
||||
# two is required; playbook/site-vars.yml in the repo
|
||||
# is never copied implicitly.
|
||||
# --site-vars FILE put FILE on CIDATA as playbook/site-vars.yml
|
||||
# (per-site values; see playbook/site-vars.example.yml)
|
||||
# --no-site-vars build without site vars (defaults; the first boot
|
||||
# picks the imaging NIC by its link). One of the two
|
||||
# is required; playbook/site-vars.yml in the repo is
|
||||
# never copied implicitly.
|
||||
# --yes do not ask for confirmation (device mode still asks
|
||||
# YES for a non-removable device)
|
||||
#
|
||||
@@ -205,8 +205,9 @@ if [ -n "$SITE_VARS" ] && [ "$NO_SITE_VARS" -eq 1 ]; then
|
||||
exit 1
|
||||
fi
|
||||
if [ -z "$SITE_VARS" ] && [ "$NO_SITE_VARS" -eq 0 ]; then
|
||||
echo "ERROR: give --site-vars FILE (the server's NIC MACs, see playbook/site-vars.example.yml)"
|
||||
echo " or --no-site-vars to build a USB whose first boot stops until they are set."
|
||||
echo "ERROR: give --site-vars FILE (per-site values, see playbook/site-vars.example.yml)"
|
||||
echo " or --no-site-vars (the first boot then picks the imaging NIC itself and uses"
|
||||
echo " the defaults for everything else)."
|
||||
exit 1
|
||||
fi
|
||||
if [ -n "$SITE_VARS" ]; then
|
||||
@@ -214,9 +215,18 @@ if [ -n "$SITE_VARS" ]; then
|
||||
echo "ERROR: --site-vars file not found: $SITE_VARS"
|
||||
exit 1
|
||||
fi
|
||||
# Both MACs are optional: an empty pxe_nic_mac makes the first boot pick
|
||||
# the imaging NIC by its link (only the imaging switch plugged in), an
|
||||
# empty corp_nic_mac makes every other port a DHCP client. One that is
|
||||
# set must be a MAC.
|
||||
for v in pxe_nic_mac corp_nic_mac; do
|
||||
if ! grep -qE "^$v:[[:space:]]*[\"']?([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}[\"']?[[:space:]]*(#.*)?$" "$SITE_VARS"; then
|
||||
echo "ERROR: $SITE_VARS does not set $v to a MAC (aa:bb:cc:dd:ee:ff)"
|
||||
line=$(grep -E "^$v:" "$SITE_VARS" | tail -1)
|
||||
[ -n "$line" ] || continue
|
||||
if grep -qE "^$v:[[:space:]]*([\"'][\"'])?[[:space:]]*(#.*)?$" <<<"$line"; then
|
||||
continue
|
||||
fi
|
||||
if ! grep -qE "^$v:[[:space:]]*[\"']?([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}[\"']?[[:space:]]*(#.*)?$" <<<"$line"; then
|
||||
echo "ERROR: $SITE_VARS sets $v to something that is not a MAC (aa:bb:cc:dd:ee:ff) or empty"
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
@@ -2,18 +2,26 @@
|
||||
# Build the installer USB as a disk image FILE with build-usb.sh image mode,
|
||||
# then upload it into the libvirt 'default' pool as volume pxetest-usb.img.
|
||||
#
|
||||
# Usage: tests/vm/build-image.sh [--with-enrollment] [--with-boot-tools] [--size SIZE]
|
||||
# Default skips enrollment/, bios-staging/, drivers-staging/ and boot-tools/
|
||||
# to keep the image small; SIZE defaults to "auto".
|
||||
# Usage: tests/vm/build-image.sh [--with-enrollment] [--with-boot-tools]
|
||||
# [--with-site-content] [--pick-nic] [--size SIZE]
|
||||
# Default skips enrollment/, bios-staging/, drivers-staging/, boot-tools/
|
||||
# and site-content/ to keep the image small; SIZE defaults to "auto".
|
||||
# --pick-nic leaves both MACs out of site-vars: the first boot picks the
|
||||
# imaging NIC by its link and every other port is a DHCP client (start the
|
||||
# server with its GE NIC unplugged: virsh domif-setlink pxetest-server
|
||||
# 52:54:00:9e:10:01 down --config).
|
||||
# Env: PXETEST_PAYLOAD_ROOT (where offline-packages/, boot-files/... live)
|
||||
source "$(dirname "$0")/common.sh"
|
||||
|
||||
SIZE=auto
|
||||
SKIPS=(--skip-enrollment --skip-bios --skip-drivers --skip-boot-tools)
|
||||
SKIPS=(--skip-enrollment --skip-bios --skip-drivers --skip-boot-tools --skip-site-content)
|
||||
PICK_NIC=0
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--with-enrollment) SKIPS=("${SKIPS[@]/--skip-enrollment}"); shift ;;
|
||||
--with-boot-tools) SKIPS=("${SKIPS[@]/--skip-boot-tools}"); shift ;;
|
||||
--with-site-content) SKIPS=("${SKIPS[@]/--skip-site-content}"); shift ;;
|
||||
--pick-nic) PICK_NIC=1; shift ;;
|
||||
--size) SIZE="$2"; shift 2 ;;
|
||||
*) die "unknown option $1" ;;
|
||||
esac
|
||||
@@ -43,13 +51,24 @@ fi
|
||||
|
||||
# Site vars for the VM "site": pin the NICs by the fixed MACs from common.sh.
|
||||
SITE_VARS="$PXETEST_CACHE/site-vars.yml"
|
||||
cat >"$SITE_VARS" <<YML
|
||||
if [ "$PICK_NIC" -eq 1 ]; then
|
||||
cat >"$SITE_VARS" <<YML
|
||||
---
|
||||
# Generated by tests/vm/build-image.sh --pick-nic for the pxetest VMs: no
|
||||
# MACs, the first boot picks the imaging NIC by its link.
|
||||
pxe_nic_mac: ""
|
||||
corp_nic_mac: ""
|
||||
YML
|
||||
log "site vars: no MACs (first boot picks the imaging NIC)"
|
||||
else
|
||||
cat >"$SITE_VARS" <<YML
|
||||
---
|
||||
# Generated by tests/vm/build-image.sh for the pxetest VMs.
|
||||
pxe_nic_mac: "$SERVER_IMAGING_MAC" # pxetest-imaging (isolated)
|
||||
corp_nic_mac: "$SERVER_GE_MAC" # libvirt default NAT ("GE/corp")
|
||||
YML
|
||||
log "site vars: pxe_nic_mac=$SERVER_IMAGING_MAC corp_nic_mac=$SERVER_GE_MAC"
|
||||
log "site vars: pxe_nic_mac=$SERVER_IMAGING_MAC corp_nic_mac=$SERVER_GE_MAC"
|
||||
fi
|
||||
|
||||
"$REPO_ROOT/scripts/build-usb.sh" --yes --payload-root "$PXETEST_PAYLOAD_ROOT" \
|
||||
--site-vars "$SITE_VARS" \
|
||||
|
||||
Reference in New Issue
Block a user