Two-NIC standard: pin imaging and GE NICs by MAC everywhere

Every server has an imaging NIC (static, the only one that serves
DHCP/TFTP/HTTP boot/SMB) and a GE corporate NIC (DHCP client, SSH and the
webapp on 9009). The playbook guessed the PXE NIC and the installer put
172.16.9.1/24 on every en* NIC, so a two-NIC box could answer on the GE
network.

- New per-site vars pxe_nic_mac / corp_nic_mac (required, from
  playbook/site-vars.yml or -e; template site-vars.example.yml), stable
  names pxe_nic_name / corp_nic_name (default pxe0 / corp0, overridable),
  pxe_server_ip, pxe_subnet, DHCP range. pxe_imaging_mac / pxe_ge_mac are
  derived from them. The play stops before touching anything if a MAC is
  unset, malformed, duplicated or not on the box (lists the box's NICs).
- Heuristic NIC detection and the single-NIC netplan task replaced by a
  two-NIC netplan template (match macaddress + set-name), installed by
  pxe-netplan-apply.sh: validate in a scratch root, back up, generate,
  rename, apply, restore the previous config on any failure. cloud-init
  network config disabled so it cannot rewrite it.
- dnsmasq: /etc/dnsmasq.d/pxe-nic-only.conf from the vars (interface=,
  except-interface=, no-dhcp-interface=); dnsmasq.conf IP/range from vars.
- UFW per interface: imaging NIC 22, 67/udp, 69/udp, 80, 4433, 445, 139,
  137/udp, 138/udp, 9009; GE NIC 22 and 9009 only. Old any-interface rules
  and the stale 9011 rule removed after the new ones are in.
- sysctl drop-in: IPv4/IPv6 forwarding off; ip_nonlocal_bind so Apache
  can bind the imaging IP before the address is up.
- Samba [global]: interfaces = 127.0.0.0/8 <imaging NIC>, bind interfaces
  only. Apache 80 and 4433 on the imaging IP only; 9009 stays on all (TODO:
  the webapp has no auth).
- autoinstall: DHCP only on en*, no static 172.16.9.1; apt geoip off and
  offline-install fallback now that the corp NIC may have a lease.
- Blancco UEFI/Secure Boot: blancco_kver (7.0.0-34-generic) drives the
  kernel, initramfs and modules builds instead of the running kernel;
  outputs vmlinuz-ubuntu-70, kexec-initrd-70.img, kmod-<kver>.tar.gz as the
  menu and blancco-init.sh expect; creates: guards keyed on the full kver.
  Ubuntu's shimx64.efi.signed published as blancco/shimx64.efi.
  download-packages.sh adds shim-signed, linux-generic-hwe-24.04 and the
  pinned linux-image/modules/modules-extra for BLANCCO_KVER.

Rendered with the live MACs and names, netplan generate output is
identical to the live config; pxe-nic-only.conf and the Samba binding
match live directive for directive.
This commit is contained in:
cproudlock
2026-10-15 08:53:41 -04:00
parent 689eebf9d1
commit ead3314f58
10 changed files with 620 additions and 133 deletions

3
.gitignore vendored
View File

@@ -126,3 +126,6 @@ reference/e10/**/*.kind1
# GE part programs carry part numbers and marking content. Keep them out of
# git; they live in ~/pxe-images/e10-testing/partprograms/
reference/e10/partprograms/
# Per-site playbook values (NIC MACs); template: playbook/site-vars.example.yml
playbook/site-vars.yml

View File

@@ -9,19 +9,31 @@ autoinstall:
variant: ""
timezone: America/New_York
# Network: static IP for isolated PXE LAN (no internet/DHCP needed)
# Network: DHCP only, on every wired NIC. On the standard two-NIC box the
# GE corporate NIC gets an address and the imaging NIC (no DHCP server on
# the imaging LAN) stays without one. No static 172.16.9.1 here: matched
# by name it would land on EVERY NIC, the GE one included, and the box
# would answer as 172.16.9.1 on the corporate network. First boot's
# playbook replaces this file with the two-NIC config pinned by MAC
# (pxe_nic_mac / corp_nic_mac in playbook/site-vars.yml). optional: the
# install and boot never wait for a lease (air-gapped is fine).
network:
version: 2
ethernets:
any-eth:
match:
name: "en*"
addresses:
- 172.16.9.1/24
dhcp4: false
dhcp4: true
dhcp6: false
optional: true
# The corp NIC may now have a lease, but the install stays offline-capable:
# no geoip lookup, and if the Ubuntu mirror is unreachable (Zscaler,
# no route) install from the ISO instead of failing.
apt:
geoip: false
fallback: offline-install
# Storage configuration
storage:
layout:

View File

@@ -20,11 +20,21 @@ contribute a `config/sites/<sitename>.yaml` template back to the repo.
### Network
Every server has two NICs: the imaging NIC (isolated imaging LAN, static
172.16.9.1, the only NIC that serves DHCP/TFTP/HTTP boot/SMB) and the GE
corporate NIC (DHCP client; SSH and the webapp on 9009 only). Put both MACs
in `playbook/site-vars.yml` (copy `playbook/site-vars.example.yml`) before
building the USB: `pxe_nic_mac`, `corp_nic_mac`. The installer brings the
network up with DHCP only; the first-boot playbook stops before changing
anything if either MAC is unset or missing, then pins both NICs by MAC in
netplan (names `pxe0` / `corp0`, overridable), dnsmasq, UFW and Samba.
| Value | Default | Where it lives |
|-------------------|----------------------|--------------------------------------------------------------------------------|
| PXE server IP | 172.16.9.1 | `playbook/pxe_server_setup.yml` (dnsmasq config, iPXE script, samba conf, webapp env), `playbook/startnet.cmd` (mount paths), `boot-tools/blancco/grub-blancco.cfg` (TFTP/HTTP URLs) |
| PXE subnet | 172.16.9.0/24 | Same as above, plus `playbook/pxe_server_setup.yml` (UFW rules) |
| DHCP range | 172.16.9.10-100 | `playbook/pxe_server_setup.yml` (dnsmasq config) |
| DHCP range | 172.16.9.10-100 | `playbook/site-vars.yml` (`pxe_dhcp_start`, `pxe_dhcp_end`) |
| NIC MACs / names | (required) / pxe0, corp0 | `playbook/site-vars.yml` (`pxe_nic_mac`, `corp_nic_mac`, `pxe_nic_name`, `corp_nic_name`) |
| Hostname | pxeserver | `autoinstall/user-data` (identity.hostname) |
### Identity and credentials

View File

@@ -25,8 +25,11 @@ The NIC guard catches those.
| `/run/pxe-nic-status.json` | last result, read by the webapp |
Sources: `playbook/pxe-server-helpers/`, installed by `pxe_server_setup.yml`
(vars `pxe_imaging_mac`, `pxe_ge_mac`, `pxe_corp_domain_suffixes`,
`pxe_nic_allowed_switches`).
(vars `pxe_nic_mac`, `corp_nic_mac`, `pxe_corp_domain_suffixes`,
`pxe_nic_allowed_switches`; `pxe_imaging_mac` / `pxe_ge_mac` are derived
from the first two). The same MACs pin the NICs in netplan
(`playbook/templates/netplan-pxe.yaml.j2`, names `pxe0` / `corp0`) and in
`/etc/dnsmasq.d/pxe-nic-only.conf`.
## How it decides

View File

@@ -0,0 +1,149 @@
#!/bin/bash
# pxe-netplan-apply.sh - install the two-NIC netplan config without ever
# leaving the box unreachable on the corporate NIC.
#
# Usage: pxe-netplan-apply.sh STAGED_YAML PXE_NAME PXE_MAC CORP_NAME CORP_MAC
#
# Called by pxe_server_setup.yml. Steps:
# 1. validate STAGED_YAML in a scratch root (netplan generate --root-dir);
# nothing on the box changes if it is invalid
# 2. stop here (prints UNCHANGED) if it is already installed, no other
# file configures wired NICs, and both NICs carry their stable names
# 3. back up /etc/netplan to /var/backups/netplan-<timestamp>/
# 4. install it (other wired configs moved aside), netplan generate
# (restore on failure)
# 5. rename the NICs to their stable names (the .link files netplan
# writes do it on every later boot), netplan apply (restore on failure)
# 6. check the imaging address and the corp NIC (warning only: the corp
# cable may simply be unplugged)
# Prints CHANGED or UNCHANGED as the last line. Exit 1 = nothing applied
# (the previous config is back in place).
set -u
if [ $# -ne 5 ]; then
echo "usage: $0 STAGED_YAML PXE_NAME PXE_MAC CORP_NAME CORP_MAC" >&2
exit 2
fi
STAGED=$1
PXE_NAME=$2
PXE_MAC=${3,,}
CORP_NAME=$4
CORP_MAC=${5,,}
NETPLAN_DIR=${NETPLAN_DIR:-/etc/netplan}
TARGET=$NETPLAN_DIR/50-cloud-init.yaml
log() { echo "pxe-netplan-apply: $*"; }
# Name of the physical NIC that has MAC $1 (empty if none).
iface_of() {
local d
for d in /sys/class/net/*; do
[ -e "$d/device" ] || continue
if [ "$(cat "$d/address" 2>/dev/null)" = "$1" ]; then
basename "$d"
return 0
fi
done
return 0
}
names_ok() {
[ "$(iface_of "$PXE_MAC")" = "$PXE_NAME" ] &&
[ "$(iface_of "$CORP_MAC")" = "$CORP_NAME" ]
}
# Other netplan files that configure wired NICs (installer leftovers, an
# en* DHCP match): they would put a second config on the two NICs. Files
# without "ethernets" (WiFi, NetworkManager wifi profiles) are left alone.
other_files() {
find "$NETPLAN_DIR" -maxdepth 1 -type f \( -name '*.yaml' -o -name '*.yml' \) \
! -path "$TARGET" -exec grep -l 'ethernets' {} + 2>/dev/null | sort
}
# 1. validate in a scratch root
SCRATCH=$(mktemp -d)
trap 'rm -rf "$SCRATCH"' EXIT
mkdir -p "$SCRATCH/etc/netplan"
install -m 600 "$STAGED" "$SCRATCH/etc/netplan/50-cloud-init.yaml"
if ! netplan generate --root-dir "$SCRATCH"; then
log "ERROR: $STAGED does not validate; nothing changed"
exit 1
fi
# 2. already in place?
if cmp -s "$STAGED" "$TARGET" && [ -z "$(other_files)" ] && names_ok; then
log "netplan already in place, $PXE_NAME and $CORP_NAME present"
echo UNCHANGED
exit 0
fi
# 3. back up
BACKUP=/var/backups/netplan-$(date +%Y%m%d-%H%M%S)
mkdir -p "$BACKUP"
cp -a "$NETPLAN_DIR/." "$BACKUP/"
log "previous config saved in $BACKUP"
restore() {
log "restoring the previous config from $BACKUP"
find "$NETPLAN_DIR" -maxdepth 1 -type f \( -name '*.yaml' -o -name '*.yml' \) -delete
cp -a "$BACKUP/." "$NETPLAN_DIR/"
netplan generate && netplan apply
}
# 4. install, moving aside the other wired configs (an en* DHCP match would
# put a DHCP client on the imaging NIC too)
other_files | while IFS= read -r f; do
log "moving aside $f (copy in $BACKUP)"
rm -f "$f"
done
install -m 600 "$STAGED" "$TARGET"
if ! netplan generate; then
log "ERROR: netplan generate failed"
restore
exit 1
fi
# 5. stable names now, so dnsmasq/UFW/Samba (which use them) work before
# the next reboot. Does not depend on the netplan version renaming.
rename() {
local mac=$1 want=$2 cur
cur=$(iface_of "$mac")
if [ -z "$cur" ]; then
log "WARNING: no NIC with MAC $mac"
return 0
fi
[ "$cur" = "$want" ] && return 0
log "renaming $cur ($mac) to $want"
if ! { ip link set dev "$cur" down &&
ip link set dev "$cur" name "$want" &&
ip link set dev "$want" up; }; then
log "WARNING: rename $cur -> $want failed; it applies at the next reboot"
ip link set dev "$cur" up 2>/dev/null
fi
return 0
}
rename "$PXE_MAC" "$PXE_NAME"
rename "$CORP_MAC" "$CORP_NAME"
if ! netplan apply; then
log "ERROR: netplan apply failed"
restore
exit 1
fi
# 6. check
for _ in $(seq 1 20); do
ip -4 -br addr show dev "$PXE_NAME" 2>/dev/null | grep -q ' [0-9]' && break
sleep 1
done
ip -br addr show dev "$PXE_NAME" 2>/dev/null ||
log "WARNING: $PXE_NAME not present yet (reboot to apply the .link names)"
for _ in $(seq 1 30); do
ip -4 -br addr show dev "$CORP_NAME" 2>/dev/null | grep -q ' [0-9]' && break
sleep 1
done
ip -4 -br addr show dev "$CORP_NAME" 2>/dev/null | grep -q ' [0-9]' ||
log "WARNING: $CORP_NAME has no IPv4 address yet (corp cable unplugged or DHCP slow)"
echo CHANGED
exit 0

View File

@@ -6,6 +6,18 @@
gather_facts: yes
pre_tasks:
# Per-site values (NIC MACs first of all). Optional file next to this
# playbook; -e still wins over it.
- name: "Look for per-site vars (playbook/site-vars.yml)"
stat:
path: "{{ playbook_dir }}/site-vars.yml"
register: site_vars_file
- name: "Load per-site vars"
include_vars:
file: "{{ playbook_dir }}/site-vars.yml"
when: site_vars_file.stat.exists
- name: "Verify required packages are installed (pre-installed from offline .debs)"
command: dpkg -s {{ item }}
loop:
@@ -22,6 +34,7 @@
- p7zip-full # 7z: Dell driver packs, catalog cabs, Blancco ISO
- zip # driver zips PESetup unpacks
- curl # Updates page downloads
- shim-signed # shimx64.efi.signed: Blancco with Secure Boot on
register: pkg_check
failed_when: false
changed_when: false
@@ -55,13 +68,40 @@
# _media/<pctype> links the base image's entries, so also look in the
# image roots (on a rebuild the _media views may not exist yet).
zscaler_ca_share_glob: "{{ samba_share }}/_media/*/Deploy/Applications/ZscalerCommercialCertificate-2048-SHA256.crt {{ samba_share }}/gea-*/Deploy/Applications/ZscalerCommercialCertificate-2048-SHA256.crt"
# --- The two NICs (site standard) ---
# Every server has two NICs: the imaging NIC (isolated imaging LAN,
# static pxe_server_ip, the ONLY NIC that serves DHCP/TFTP/HTTP boot/SMB)
# and the GE corporate NIC (DHCP client; SSH and the webapp on 9009 for
# admins, outbound updates through Zscaler). Both MACs are REQUIRED:
# set them in playbook/site-vars.yml on the USB (copy
# site-vars.example.yml) or with -e. The playbook stops before it
# touches anything if either is unset or not on this box. Read them
# with: ip -br link
pxe_nic_mac: ""
corp_nic_mac: ""
# Stable names, set by netplan (match macaddress + set-name). dnsmasq,
# UFW and Samba refer to these. A site may keep its existing names
# (e.g. pxe_nic_name: enx34c8d6b11010, corp_nic_name: enp0s31f6).
pxe_nic_name: "pxe0"
corp_nic_name: "corp0"
# Imaging LAN addressing. NOTE: 172.16.9.1 is also hard-coded in the
# iPXE menu, startnet.cmd, blancco-preferences.xml and the unattend
# files (see docs/SITE-CUSTOMIZATION.md); change those together.
pxe_server_ip: "172.16.9.1"
pxe_subnet: "172.16.9.0/24"
pxe_prefix: "{{ pxe_subnet.split('/')[1] }}"
pxe_dhcp_start: "172.16.9.10"
pxe_dhcp_end: "172.16.9.100"
pxe_dhcp_lease: "12h"
# Route metric for the corp NIC's DHCP default route.
corp_route_metric: 100
# NIC guard (pxe-nic-check): dnsmasq must never serve DHCP/PXE on the GE
# corporate network. Pin the two NICs by MAC (ip -br link); left blank,
# the imaging NIC is the interface= of /etc/dnsmasq.d/pxe-nic-only.conf
# or the holder of 172.16.9.1, and the GE NIC is the one with a DHCP
# lease. A MAC of a bridge port resolves to the bridge (br-pxe).
pxe_imaging_mac: ""
pxe_ge_mac: ""
# corporate network. These are derived from pxe_nic_mac / corp_nic_mac
# (one source of truth) and written to /etc/pxe-server/nic.conf; do not
# set them separately (the playbook fails if they disagree). A MAC of a
# bridge port resolves to the bridge.
pxe_imaging_mac: "{{ pxe_nic_mac }}"
pxe_ge_mac: "{{ corp_nic_mac }}"
# A foreign DHCP server handing out one of these domains (whole labels:
# wjs.geaerospace.net matches, notgeaerospace.net does not) = GE network.
pxe_corp_domain_suffixes: "geaerospace.net, ge.com"
@@ -69,6 +109,35 @@
# LLDP system name / CDP device id / chassis MAC, comma separated, so
# it does not count as a corporate switch. 802.1X always counts.
pxe_nic_allowed_switches: ""
# UFW, per interface (see the UFW tasks). Imaging NIC: SSH, DHCP, TFTP,
# HTTP boot, iPXE menu, SMB (+ NetBIOS for older clients), webapp.
ufw_pxe_rules:
- { port: "22", proto: tcp }
- { port: "67", proto: udp }
- { port: "69", proto: udp }
- { port: "80", proto: tcp }
- { port: "4433", proto: tcp }
- { port: "445", proto: tcp }
- { port: "139", proto: tcp }
- { port: "137", proto: udp }
- { port: "138", proto: udp }
- { port: "9009", proto: tcp }
# GE corporate NIC: SSH and the webapp, nothing else.
ufw_corp_rules:
- { port: "22", proto: tcp }
- { port: "9009", proto: tcp }
# --- Blancco UEFI / Secure Boot path ---
# Blancco's UEFI entries boot a Canonical-signed Ubuntu HWE kernel
# (through Ubuntu's shim when Secure Boot is on), then switch_root into
# the Blancco rootfs. The kernel, its modules and the initramfs are
# built from this version, which must be installed on the box
# (linux-generic-hwe-24.04 from the offline packages; see
# scripts/download-packages.sh BLANCCO_KVER). The file suffix is
# major+minor ("7.0.0-34-generic" -> "70"), as GetPxeScript.aspx expects:
# vmlinuz-ubuntu-70, kexec-initrd-70.img; modules are
# kmod-<blancco_kver>.tar.gz (blancco-init.sh fetches kmod-$(uname -r)).
blancco_kver: "7.0.0-34-generic"
blancco_ksuffix: "{{ blancco_kver.split('.')[0] }}{{ blancco_kver.split('.')[1] }}"
image_types:
- gea-standard
- gea-engineer
@@ -91,50 +160,107 @@
- Tools
tasks:
- name: "Gather minimal network facts"
ansible.builtin.setup:
filter:
- ansible_interfaces
- ansible_default_ipv4
# --- The two NICs: validate, then pin by MAC (netplan) ---
# Replaces the old guess (first en* without 172.16.9.1 / the default
# route), which on a two-NIC box could pick the GE NIC.
- name: "NICs: check the per-site NIC vars"
assert:
that:
- pxe_nic_mac | length > 0
- corp_nic_mac | length > 0
- pxe_nic_mac is match('^([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}$')
- corp_nic_mac is match('^([0-9A-Fa-f]{2}:){5}[0-9A-Fa-f]{2}$')
- pxe_nic_mac | lower != corp_nic_mac | lower
- pxe_imaging_mac | lower == pxe_nic_mac | lower
- pxe_ge_mac | lower == corp_nic_mac | lower
- pxe_nic_name | length > 0 and pxe_nic_name | length <= 15
- corp_nic_name | length > 0 and corp_nic_name | length <= 15
- pxe_nic_name != corp_nic_name
fail_msg: >-
Set pxe_nic_mac (imaging NIC) and corp_nic_mac (GE corporate NIC)
to two different MACs (aa:bb:cc:dd:ee:ff) in playbook/site-vars.yml
(copy site-vars.example.yml) or with -e, then re-run. Do not set
pxe_imaging_mac / pxe_ge_mac yourself; they follow these. Names
(pxe_nic_name, corp_nic_name) must differ and be 1-15 chars.
Now: pxe_nic_mac='{{ pxe_nic_mac }}' corp_nic_mac='{{ corp_nic_mac }}'.
quiet: yes
- name: "Bring up all ethernet-like interfaces"
command: ip link set dev {{ item }} up
loop: "{{ ansible_interfaces | select('match','^e(th|n)') | list }}"
ignore_errors: yes
- name: "NICs: list the physical NICs on this box"
shell: |
for d in /sys/class/net/*; do
[ -e "$d/device" ] || continue
echo "$(basename "$d") $(cat "$d/address")"
done
register: nic_list
changed_when: false
- name: "Find interface with 172.16.9.1 already configured"
set_fact:
preconfigured_iface: >-
{{ ansible_interfaces
| select('match','^e(th|n)')
| map('regex_replace','^(.*)$','ansible_\1')
| map('extract', hostvars[inventory_hostname])
| selectattr('ipv4','defined')
| selectattr('ipv4.address','equalto','172.16.9.1')
| map(attribute='device')
| list
| first
| default('') }}
ignore_errors: yes
- name: "NICs: both MACs must be on this box"
vars:
box_macs: "{{ nic_list.stdout_lines | map('regex_replace', '^.* ', '') | map('lower') | list }}"
assert:
that:
- pxe_nic_mac | lower in box_macs
- corp_nic_mac | lower in box_macs
fail_msg: >-
{{ 'pxe_nic_mac ' ~ pxe_nic_mac ~ ' is not on this box. ' if pxe_nic_mac | lower not in box_macs else '' }}{{ 'corp_nic_mac ' ~ corp_nic_mac ~ ' is not on this box. ' if corp_nic_mac | lower not in box_macs else '' }}Nothing was changed (the network config would have left that NIC
unconfigured). Physical NICs here (name MAC):
{{ nic_list.stdout_lines | join('; ') }}.
Fix playbook/site-vars.yml and re-run.
quiet: yes
- name: "Determine PXE interface"
set_fact:
pxe_iface: >-
{{ preconfigured_iface | default('',true)
or (ansible_interfaces
| select('match','^e(th|n)')
| reject('equalto','lo')
| reject('equalto', ansible_default_ipv4.interface | default(''))
| list
)
| first
| default(ansible_default_ipv4.interface | default(
ansible_interfaces | select('match','^e(th|n)') | first | default('eth0')
)) }}
- name: "NICs: stop cloud-init from rewriting the network config"
copy:
dest: /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
mode: '0644'
content: |
# Managed by pxe_server_setup.yml: /etc/netplan/50-cloud-init.yaml
# is the playbook's two-NIC config, not cloud-init's.
network: {config: disabled}
- name: "Debug: final pxe_iface choice"
- name: "NICs: deploy the netplan apply helper"
copy:
src: "{{ usb_mount }}/pxe-server-helpers/pxe-netplan-apply.sh"
dest: /usr/local/sbin/pxe-netplan-apply.sh
mode: '0755'
- name: "NICs: stage the two-NIC netplan config"
template:
src: templates/netplan-pxe.yaml.j2
dest: /etc/pxe-server-netplan.yaml
mode: '0600'
- name: "NICs: validate, install and apply netplan (rolls back on failure)"
command: >-
/usr/local/sbin/pxe-netplan-apply.sh /etc/pxe-server-netplan.yaml
{{ pxe_nic_name }} {{ pxe_nic_mac | lower }}
{{ corp_nic_name }} {{ corp_nic_mac | lower }}
register: netplan_apply
changed_when: netplan_apply.stdout_lines | last | default('') == 'CHANGED'
- name: "NICs: result"
debug:
msg: "Using {{ pxe_iface }} for DHCP/TFTP"
msg: "{{ netplan_apply.stdout_lines }}"
- name: "Pin IP forwarding off (the imaging LAN is never routed)"
copy:
dest: /etc/sysctl.d/99-pxe-net.conf
mode: '0644'
content: |
# Managed by pxe_server_setup.yml. The server never routes between
# the imaging LAN and the GE corporate network.
net.ipv4.ip_forward = 0
net.ipv6.conf.all.forwarding = 0
net.ipv6.conf.default.forwarding = 0
# Apache listens on {{ pxe_server_ip }} only (ports 80/4433). Let it
# bind before networkd has put the address on {{ pxe_nic_name }} (no
# carrier at boot, or apache starting first), or apache2 fails to
# start and takes the webapp on 9009 down with it.
net.ipv4.ip_nonlocal_bind = 1
register: pxe_net_sysctl
- name: "Apply the network sysctls now (Apache restarts before the handlers)"
command: sysctl -p /etc/sysctl.d/99-pxe-net.conf
when: pxe_net_sysctl is changed
- name: "Expand root partition and filesystem to use full disk"
args:
@@ -175,9 +301,11 @@
backup: yes
content: |
port=0
listen-address=172.16.9.1
listen-address={{ pxe_server_ip }}
bind-dynamic
dhcp-range=172.16.9.10,172.16.9.100,12h
# Interface pinning (serve {{ pxe_nic_name }} only, never
# {{ corp_nic_name }}) is in /etc/dnsmasq.d/pxe-nic-only.conf.
dhcp-range={{ pxe_dhcp_start }},{{ pxe_dhcp_end }},{{ pxe_dhcp_lease }}
# No default gateway (option 3) and no DNS (option 6) handed out:
# the PXE network is isolated and the PXE server does not forward
# internet traffic. Previously we set both, which made imaged PCs
@@ -193,7 +321,7 @@
# Important: dnsmasq DEFAULTS to sending its own listening address as
# both router and DNS when these options are unset. Commenting them
# out is NOT the same as disabling - imaged PCs (and Blancco PXE
# clients) end up with 172.16.9.1 as gateway. The empty-value form
# clients) end up with {{ pxe_server_ip }} as gateway. The empty-value form
# below explicitly suppresses both options.
dhcp-option=3
dhcp-option=6
@@ -211,7 +339,7 @@
dhcp-vendorclass=set:bios,PXEClient:Arch:00000
dhcp-userclass=set:ipxe,iPXE
dhcp-boot=tag:bios,tag:!ipxe,ipxe.pxe
dhcp-boot=tag:ipxe,http://172.16.9.1:4433/Altiris/iPXE/GetPxeScript.aspx
dhcp-boot=tag:ipxe,http://{{ pxe_server_ip }}:4433/Altiris/iPXE/GetPxeScript.aspx
dhcp-boot=tag:!bios,tag:!ipxe,ipxe-shim.efi
log-dhcp
# Per-lease state cleanup: flush conntrack + port-445 sockets for
@@ -225,6 +353,13 @@
# firmware / iPXE requests.
script-on-renewal
- name: "Pin dnsmasq to the imaging NIC (never DHCP/PXE on the GE NIC)"
template:
src: templates/dnsmasq-pxe-nic-only.conf.j2
dest: /etc/dnsmasq.d/pxe-nic-only.conf
mode: '0644'
backup: yes
# --- NIC guard: keep dnsmasq off the GE corporate network ---
# pxe-nic-check classifies each NIC from what it hears (LLDP/CDP/802.1X
# from managed switches, foreign DHCP offers and their domain). The
@@ -250,22 +385,14 @@
dest: /etc/pxe-server/nic.conf
mode: '0644'
content: |
# Managed by pxe_server_setup.yml (vars pxe_imaging_mac, pxe_ge_mac,
# pxe_corp_domain_suffixes). Read by /usr/local/sbin/pxe-nic-check.
# Managed by pxe_server_setup.yml (vars pxe_nic_mac, corp_nic_mac,
# pxe_server_ip, pxe_corp_domain_suffixes). Read by
# /usr/local/sbin/pxe-nic-check.
[nic]
{% if pxe_imaging_mac %}
imaging_mac = {{ pxe_imaging_mac | lower }}
{% else %}
# imaging_mac unset: falls back to interface= in
# /etc/dnsmasq.d/pxe-nic-only.conf, then the holder of own_ip
{% endif %}
{% if pxe_ge_mac %}
ge_mac = {{ pxe_ge_mac | lower }}
{% else %}
# ge_mac unset: falls back to the interface holding a DHCP lease
{% endif %}
corp_domain_suffixes = {{ pxe_corp_domain_suffixes }}
own_ip = 172.16.9.1
own_ip = {{ pxe_server_ip }}
allowed_switches = {{ pxe_nic_allowed_switches }}
- name: "NIC guard: create dnsmasq drop-in directory"
@@ -344,10 +471,22 @@
mode: '0644'
backup: yes
- name: "Ensure Apache listens on port 4433"
# Boot files (80) and the iPXE menu (4433) are served on the imaging IP
# only, never on the GE NIC. 9009 (webapp) stays on all addresses: admins
# reach it from the GE side (see pxe-server.conf below).
- name: "Apache: serve port 80 on the imaging IP only"
lineinfile:
path: /etc/apache2/ports.conf
line: "Listen 4433"
regexp: '^\s*Listen\s+(\S+:)?80\s*$'
line: "Listen {{ pxe_server_ip }}:80"
backup: yes
state: present
- name: "Apache: serve port 4433 (iPXE menu) on the imaging IP only"
lineinfile:
path: /etc/apache2/ports.conf
regexp: '^\s*Listen\s+(\S+:)?4433\s*$'
line: "Listen {{ pxe_server_ip }}:4433"
backup: yes
state: present
@@ -619,6 +758,23 @@
# or WinPE clients cannot follow them.
allow insecure wide links = yes
# Runs after the symlinks block so it lands first under [global], as on
# the live server.
- name: "Bind Samba to the imaging NIC only"
blockinfile:
path: /etc/samba/smb.conf
backup: yes
marker: "# {mark} MANAGED - PXE NIC BIND"
insertafter: '\[global\]'
block: |
# Bind Samba to the PXE LAN only. The second NIC reaches the GE corporate
# network, and these shares (blancco-reports, enrollment, image-upload,
# clonezilla) use shared, well-known imaging passwords and are writable by
# design. Without this they are offered to the corporate network the moment
# the cable is plugged in.
interfaces = 127.0.0.0/8 {{ pxe_nic_name }}
bind interfaces only = yes
- name: "Samba SMB session handling for WinPE re-image robustness"
blockinfile:
path: /etc/samba/smb.conf
@@ -876,8 +1032,37 @@
# --- Blancco PXE boot via Ubuntu kernel + switch_root ---
# Blancco's own kernel freezes on Dell Precision towers. Workaround:
# Boot Ubuntu kernel, download Blancco rootfs, overlay mount, switch_root.
# Kernel, modules and initramfs all come from blancco_kver (the HWE
# kernel), not the server's running kernel: the server may run 6.8 while
# Blancco boots 7.0. Output names carry the version so a changed
# blancco_kver rebuilds instead of being skipped by the creates: guards.
- name: "Build Blancco PXE initramfs"
- name: "Blancco: is the Blancco kernel ({{ blancco_kver }}) installed?"
stat:
path: "{{ item }}"
loop:
- "/boot/vmlinuz-{{ blancco_kver }}"
- "/lib/modules/{{ blancco_kver }}/modules.dep"
register: blancco_kernel_files
- name: "Blancco: kernel check result"
set_fact:
blancco_kernel_ok: "{{ blancco_kernel_files.results | map(attribute='stat.exists') | min }}"
- name: "Blancco: warn when the Blancco kernel is missing"
debug:
msg: >-
WARNING: /boot/vmlinuz-{{ blancco_kver }} or
/lib/modules/{{ blancco_kver }} is missing, so the Blancco UEFI
files (vmlinuz-ubuntu-{{ blancco_ksuffix }},
kexec-initrd-{{ blancco_ksuffix }}.img,
kmod-{{ blancco_kver }}.tar.gz) are NOT built. Install
linux-image-{{ blancco_kver }} + linux-modules-extra-{{ blancco_kver }}
(scripts/download-packages.sh, BLANCCO_KVER) or set blancco_kver
to an installed version (ls /lib/modules), then re-run.
when: not blancco_kernel_ok
- name: "Build Blancco PXE initramfs (kexec-initrd-{{ blancco_ksuffix }}.img)"
# The narrow hand-picked NIC driver list used before 2026-04-22 produced
# a 2 MB initramfs that hung on "waiting for network interface" for any
# hardware outside e1000e/igb/tg3/bnx2/bnxt_en/b44. This rewrite sweeps
@@ -889,11 +1074,15 @@
# the narrow build cost us when a NIC was unsupported.
args:
executable: /bin/bash
creates: "{{ web_root }}/blancco/kexec-initrd.img"
# Stamp named after the full version: 7.0.0-34 -> 7.0.0-40 keeps the
# file name (suffix 70) but must rebuild.
creates: "{{ web_root }}/blancco/.kexec-initrd-{{ blancco_ksuffix }}.{{ blancco_kver }}.built"
when: blancco_kernel_ok
shell: |
set -e
WORK=$(mktemp -d)
KVER=$(uname -r)
KVER="{{ blancco_kver }}"
OUT="{{ web_root }}/blancco/kexec-initrd-{{ blancco_ksuffix }}.img"
mkdir -p "$WORK"/{bin,lib/modules/$KVER/kernel,lib64,sbin,usr/share/udhcpc,etc,run,proc,sys,dev}
# Busybox (static) - bundled on USB at playbook/busybox-static
@@ -979,18 +1168,47 @@
# Build CPIO
cd "$WORK"
find . | cpio -o -H newc --quiet | gzip -1 > "{{ web_root }}/blancco/kexec-initrd.img"
# Build beside the target and rename: a PC may be booting the old one.
find . | cpio -o -H newc --quiet | gzip -1 > "$OUT.tmp"
mv -f "$OUT.tmp" "$OUT"
cd /
rm -rf "$WORK"
echo "Built kexec-initrd.img: $(stat -c %s '{{ web_root }}/blancco/kexec-initrd.img') bytes"
rm -f "{{ web_root }}"/blancco/.kexec-initrd-{{ blancco_ksuffix }}.*.built
touch "{{ web_root }}/blancco/.kexec-initrd-{{ blancco_ksuffix }}.{{ blancco_kver }}.built"
echo "Built $(basename "$OUT") for $KVER: $(stat -c %s "$OUT") bytes"
ignore_errors: yes
- name: "Copy Ubuntu kernel for Blancco PXE boot"
- name: "Copy Ubuntu kernel for Blancco PXE boot (vmlinuz-ubuntu-{{ blancco_ksuffix }})"
copy:
src: "/boot/vmlinuz-{{ ansible_kernel }}"
dest: "{{ web_root }}/blancco/vmlinuz-ubuntu"
src: "/boot/vmlinuz-{{ blancco_kver }}"
dest: "{{ web_root }}/blancco/vmlinuz-ubuntu-{{ blancco_ksuffix }}"
remote_src: yes
mode: '0644'
when: blancco_kernel_ok
# Secure Boot on: iPXE 2.0 starts the Canonical-signed kernel above
# through Ubuntu's Microsoft-signed shim (GetPxeScript.aspx: shim
# http://${server}/blancco/shimx64.efi). From the shim-signed package.
- name: "Blancco: find Ubuntu's signed shim"
stat:
path: /usr/lib/shim/shimx64.efi.signed
register: ubuntu_shim
- name: "Blancco: publish Ubuntu's signed shim as blancco/shimx64.efi"
copy:
src: /usr/lib/shim/shimx64.efi.signed
dest: "{{ web_root }}/blancco/shimx64.efi"
remote_src: yes
mode: '0644'
when: ubuntu_shim.stat.exists
- name: "Blancco: warn when the signed shim is missing"
debug:
msg: >-
WARNING: /usr/lib/shim/shimx64.efi.signed not found (package
shim-signed). Blancco with Secure Boot ON will not start; with
Secure Boot off it still boots (the menu's shim line fails over).
when: not ubuntu_shim.stat.exists
- name: "Create TFTP blancco directory"
file:
@@ -1015,10 +1233,12 @@
- amd-ucode.img
- config.img
- name: "Build Ubuntu kernel modules tarball for Blancco"
- name: "Build Ubuntu kernel modules tarball for Blancco (kmod-{{ blancco_kver }}.tar.gz)"
when: blancco_kernel_ok
shell: |
set -e
KVER=$(uname -r)
KVER="{{ blancco_kver }}"
OUT="{{ web_root }}/blancco/kmod-$KVER.tar.gz"
# Ship the host's iwlwifi firmware with its modules. The UEFI path
# runs this Ubuntu kernel on Blancco's rootfs, and Blancco 7.20 only
# carries API 89 ucode for AX201/AX210/AX211, which the 6.8 iwlwifi
@@ -1026,10 +1246,12 @@
# only, no lib/ entry: on the Blancco rootfs lib is a symlink to
# usr/lib and a directory entry would replace it.
cd /
tar czf "{{ web_root }}/blancco/kmod.tar.gz" -C / "lib/modules/$KVER" $(ls -d lib/firmware/iwlwifi-*)
echo "Built kmod.tar.gz: $(du -h '{{ web_root }}/blancco/kmod.tar.gz' | cut -f1)"
# blancco-init.sh fetches kmod-$(uname -r).tar.gz (the booted kernel).
tar czf "$OUT.tmp" -C / "lib/modules/$KVER" $(ls -d lib/firmware/iwlwifi-*)
mv -f "$OUT.tmp" "$OUT"
echo "Built $(basename "$OUT"): $(du -h "$OUT" | cut -f1)"
args:
creates: "{{ web_root }}/blancco/kmod.tar.gz"
creates: "{{ web_root }}/blancco/kmod-{{ blancco_kver }}.tar.gz"
- name: "Extract config-clean.xml from config.img (strip null bytes)"
shell: |
@@ -1068,7 +1290,7 @@
shell: |
set -e
python3 -c 'import xml.etree.ElementTree as ET; ET.parse("{{ web_root }}/blancco/preferences.xml")'
grep -q '<hostname>172.16.9.1</hostname>' "{{ web_root }}/blancco/preferences.xml"
grep -q '<hostname>{{ pxe_server_ip }}</hostname>' "{{ web_root }}/blancco/preferences.xml"
grep -q '<path>blancco-reports</path>' "{{ web_root }}/blancco/preferences.xml"
changed_when: false
@@ -1099,19 +1321,56 @@
- apache2
- smbd
- name: "Allow necessary firewall ports (UFW)"
# Firewall per interface. Imaging NIC: everything imaging needs. GE
# corporate NIC: SSH and the webapp only. New rules go in first, then the
# old any-interface rules are removed, so SSH is never closed mid-run.
- name: "UFW: allow imaging services on the imaging NIC only"
ufw:
rule: allow
port: "{{ item }}"
proto: "{{ 'udp' if item in ['67','69'] else 'tcp' }}"
direction: in
interface: "{{ pxe_nic_name }}"
port: "{{ item.port }}"
proto: "{{ item.proto }}"
loop: "{{ ufw_pxe_rules }}"
loop_control:
label: "{{ pxe_nic_name }} {{ item.port }}/{{ item.proto }}"
- name: "UFW: allow SSH and the webapp on the GE corporate NIC"
ufw:
rule: allow
direction: in
interface: "{{ corp_nic_name }}"
port: "{{ item.port }}"
proto: "{{ item.proto }}"
loop: "{{ ufw_corp_rules }}"
loop_control:
label: "{{ corp_nic_name }} {{ item.port }}/{{ item.proto }}"
- name: "UFW: remove the old any-interface rules"
ufw:
rule: allow
port: "{{ item.port }}"
proto: "{{ item.proto }}"
delete: yes
loop:
- "22"
- "67"
- "69"
- "80"
- "4433"
- "445"
- "9009"
- { port: "22", proto: tcp }
- { port: "67", proto: udp }
- { port: "69", proto: udp }
- { port: "80", proto: tcp }
- { port: "4433", proto: tcp }
- { port: "445", proto: tcp }
- { port: "9009", proto: tcp }
loop_control:
label: "any {{ item.port }}/{{ item.proto }}"
- name: "UFW: remove the stale 9011 rule (old webapp port)"
ufw:
rule: allow
direction: in
interface: "{{ pxe_nic_name }}"
port: "9011"
proto: tcp
delete: yes
- name: "Enable UFW firewall"
ufw:
@@ -1302,6 +1561,10 @@
copy:
dest: /etc/apache2/sites-available/pxe-server.conf
content: |
# 9009 listens on every address: admins use the webapp from the GE
# corporate NIC. TODO: the webapp has NO authentication; anyone on
# the GE network who can reach this box can drive it. Add auth (or
# restrict 9009 on the corp NIC to an admin subnet in UFW).
Listen 9009
<VirtualHost *:80>
DocumentRoot {{ web_root }}
@@ -1346,46 +1609,11 @@
name: apache2
state: reloaded
# Single-NIC fresh-deploy default. Boxes that need higher throughput
# (e.g. WJF prod uses a USB-C 5 Gbps NIC) override this with a bridge
# config bonding the USB NIC + onboard NIC into br-pxe. Live override
# currently deployed on 172.16.9.1 (do NOT re-run this task there
# without first reviewing /etc/netplan/50-cloud-init.yaml.pre-gold-swap):
#
# network:
# version: 2
# renderer: networkd
# ethernets:
# enp128s31f6: { dhcp4: no }
# enx34c8d6b11010: { dhcp4: no }
# bridges:
# br-pxe:
# interfaces: [enp128s31f6, enx34c8d6b11010]
# addresses: [172.16.9.1/24]
# parameters:
# stp: false
#
# Re-running THIS task on a box already using the bridge config will
# replace the bridge with a single-iface flat config and likely break
# the PXE LAN. Skip this task or restore the .pre-gold-swap backup
# before applying.
- name: "Configure static IP for PXE interface"
copy:
dest: /etc/netplan/50-cloud-init.yaml
backup: yes
content: |
network:
version: 2
renderer: networkd
ethernets:
{{ pxe_iface }}:
dhcp4: no
addresses: [172.16.9.1/24]
notify: "Apply netplan"
# The network config (two NICs pinned by MAC) is applied near the top of
# the play, "NICs: ..." tasks, so dnsmasq/UFW/Samba see the final names.
# The old br-pxe bridge layout (onboard + USB NIC bonded) is gone: one NIC
# per network, see templates/netplan-pxe.yaml.j2.
handlers:
- name: "Apply netplan"
command: netplan apply
- name: "reload sysctl"
command: sysctl --system

View File

@@ -0,0 +1,31 @@
---
# Per-site values for pxe_server_setup.yml. Copy to playbook/site-vars.yml
# (next to the playbook, so it rides on the USB) and fill in. The playbook
# loads it on first boot; -e on the command line still overrides it.
# site-vars.yml is git-ignored: each site's MACs stay with that site.
#
# REQUIRED: the two NICs, by MAC (ip -br link on the box, or the label).
# imaging NIC = cable to the isolated imaging switch (serves DHCP/PXE/SMB)
# corp NIC = cable to the GE corporate network (SSH + webapp 9009 only)
# The playbook changes nothing on the network until both are set, differ,
# and are both present on the box.
pxe_nic_mac: "" # e.g. "34:c8:d6:b1:10:10"
corp_nic_mac: "" # e.g. "20:88:10:af:0d:e5"
# Optional: stable interface names (default pxe0 / corp0). Keep a box's
# existing names instead, e.g. on the WJ server:
# pxe_nic_name: enx34c8d6b11010
# corp_nic_name: enp0s31f6
# Optional: imaging LAN addressing (defaults shown). 172.16.9.1 is also
# hard-coded in the iPXE menu, startnet.cmd, blancco-preferences.xml and the
# unattend files; see docs/SITE-CUSTOMIZATION.md before changing it.
# pxe_server_ip: "172.16.9.1"
# pxe_subnet: "172.16.9.0/24"
# pxe_dhcp_start: "172.16.9.10"
# pxe_dhcp_end: "172.16.9.100"
# pxe_dhcp_lease: "12h"
# Optional: kernel the Blancco UEFI entries boot (must be installed; see
# scripts/download-packages.sh BLANCCO_KVER).
# blancco_kver: "7.0.0-34-generic"

View File

@@ -0,0 +1,8 @@
# Managed by pxe_server_setup.yml (vars pxe_nic_name, corp_nic_name).
# Pin dnsmasq to the imaging NIC. Without it the box offers PXE/DHCP on the
# GE corporate LAN the moment that cable is plugged in. Works together with
# port=0 / listen-address / bind-dynamic in /etc/dnsmasq.conf and the
# pxe-nic-check guard (docs/pxe-nic-guard.md).
interface={{ pxe_nic_name }}
except-interface={{ corp_nic_name }}
no-dhcp-interface={{ corp_nic_name }}

View File

@@ -0,0 +1,28 @@
# Managed by pxe_server_setup.yml (vars pxe_nic_mac, corp_nic_mac,
# pxe_nic_name, corp_nic_name, pxe_server_ip, pxe_subnet). Local edits are
# replaced on the next playbook run; the previous file is kept under
# /var/backups/netplan-<timestamp>/.
#
# Two NICs, each matched by MAC and given a stable name:
# pxe-lan imaging LAN, static, no gateway, no DNS. The ONLY NIC that
# serves DHCP/TFTP/HTTP boot/SMB.
# ge-corp GE corporate network, DHCP client (SSH + webapp 9009, updates).
network:
version: 2
renderer: networkd
ethernets:
pxe-lan:
match:
macaddress: "{{ pxe_nic_mac | lower }}"
set-name: {{ pxe_nic_name }}
dhcp4: false
dhcp6: false
addresses:
- {{ pxe_server_ip }}/{{ pxe_prefix }}
ge-corp:
match:
macaddress: "{{ corp_nic_mac | lower }}"
set-name: {{ corp_nic_name }}
dhcp4: true
dhcp4-overrides:
route-metric: {{ corp_route_metric }}

View File

@@ -37,6 +37,7 @@ if [ "${IN_DOCKER:-}" != "1" ] && [ "$HOST_CODENAME" != "noble" ]; then
-v "$REPO_DIR:/repo" \
-v "$OUT_DIR_ABS:/out" \
-e IN_DOCKER=1 \
-e BLANCCO_KVER="${BLANCCO_KVER:-}" \
-w /repo \
ubuntu:24.04 \
bash -c "apt-get update -qq && apt-get install -y --no-install-recommends sudo python3-pip python3-setuptools python3-wheel ca-certificates >/dev/null && /repo/scripts/download-packages.sh /out"
@@ -50,6 +51,14 @@ fi
mkdir -p "$OUT_DIR"
# Kernel the Blancco UEFI path boots (playbook var blancco_kver; keep the two
# in step). The playbook builds vmlinuz-ubuntu-70, kexec-initrd-70.img and
# kmod-<kver>.tar.gz from this kernel's /boot and /lib/modules, and the
# Canonical-signed image is what Ubuntu's shim starts with Secure Boot on.
# The versioned packages pin it; the HWE metapackage keeps the box on the
# HWE line. Override: BLANCCO_KVER=7.0.0-40-generic ./download-packages.sh
BLANCCO_KVER="${BLANCCO_KVER:-7.0.0-34-generic}"
# Packages installed by the Ansible playbook (pxe_server_setup.yml)
PLAYBOOK_PACKAGES=(
ansible
@@ -71,6 +80,12 @@ PLAYBOOK_PACKAGES=(
sbsigntool
zip
curl
# Blancco with Secure Boot on: shimx64.efi.signed + the HWE kernel
shim-signed
linux-generic-hwe-24.04
"linux-image-${BLANCCO_KVER}"
"linux-modules-${BLANCCO_KVER}"
"linux-modules-extra-${BLANCCO_KVER}"
)
# Packages installed during autoinstall late-commands (NetworkManager, WiFi, etc.)