Two-NIC standard: pin imaging and GE NICs by MAC everywhere

Every server has an imaging NIC (static, the only one that serves
DHCP/TFTP/HTTP boot/SMB) and a GE corporate NIC (DHCP client, SSH and the
webapp on 9009). The playbook guessed the PXE NIC and the installer put
172.16.9.1/24 on every en* NIC, so a two-NIC box could answer on the GE
network.

- New per-site vars pxe_nic_mac / corp_nic_mac (required, from
  playbook/site-vars.yml or -e; template site-vars.example.yml), stable
  names pxe_nic_name / corp_nic_name (default pxe0 / corp0, overridable),
  pxe_server_ip, pxe_subnet, DHCP range. pxe_imaging_mac / pxe_ge_mac are
  derived from them. The play stops before touching anything if a MAC is
  unset, malformed, duplicated or not on the box (lists the box's NICs).
- Heuristic NIC detection and the single-NIC netplan task replaced by a
  two-NIC netplan template (match macaddress + set-name), installed by
  pxe-netplan-apply.sh: validate in a scratch root, back up, generate,
  rename, apply, restore the previous config on any failure. cloud-init
  network config disabled so it cannot rewrite it.
- dnsmasq: /etc/dnsmasq.d/pxe-nic-only.conf from the vars (interface=,
  except-interface=, no-dhcp-interface=); dnsmasq.conf IP/range from vars.
- UFW per interface: imaging NIC 22, 67/udp, 69/udp, 80, 4433, 445, 139,
  137/udp, 138/udp, 9009; GE NIC 22 and 9009 only. Old any-interface rules
  and the stale 9011 rule removed after the new ones are in.
- sysctl drop-in: IPv4/IPv6 forwarding off; ip_nonlocal_bind so Apache
  can bind the imaging IP before the address is up.
- Samba [global]: interfaces = 127.0.0.0/8 <imaging NIC>, bind interfaces
  only. Apache 80 and 4433 on the imaging IP only; 9009 stays on all (TODO:
  the webapp has no auth).
- autoinstall: DHCP only on en*, no static 172.16.9.1; apt geoip off and
  offline-install fallback now that the corp NIC may have a lease.
- Blancco UEFI/Secure Boot: blancco_kver (7.0.0-34-generic) drives the
  kernel, initramfs and modules builds instead of the running kernel;
  outputs vmlinuz-ubuntu-70, kexec-initrd-70.img, kmod-<kver>.tar.gz as the
  menu and blancco-init.sh expect; creates: guards keyed on the full kver.
  Ubuntu's shimx64.efi.signed published as blancco/shimx64.efi.
  download-packages.sh adds shim-signed, linux-generic-hwe-24.04 and the
  pinned linux-image/modules/modules-extra for BLANCCO_KVER.

Rendered with the live MACs and names, netplan generate output is
identical to the live config; pxe-nic-only.conf and the Samba binding
match live directive for directive.
This commit is contained in:
cproudlock
2026-10-15 08:53:41 -04:00
parent 689eebf9d1
commit ead3314f58
10 changed files with 620 additions and 133 deletions

View File

@@ -20,11 +20,21 @@ contribute a `config/sites/<sitename>.yaml` template back to the repo.
### Network
Every server has two NICs: the imaging NIC (isolated imaging LAN, static
172.16.9.1, the only NIC that serves DHCP/TFTP/HTTP boot/SMB) and the GE
corporate NIC (DHCP client; SSH and the webapp on 9009 only). Put both MACs
in `playbook/site-vars.yml` (copy `playbook/site-vars.example.yml`) before
building the USB: `pxe_nic_mac`, `corp_nic_mac`. The installer brings the
network up with DHCP only; the first-boot playbook stops before changing
anything if either MAC is unset or missing, then pins both NICs by MAC in
netplan (names `pxe0` / `corp0`, overridable), dnsmasq, UFW and Samba.
| Value | Default | Where it lives |
|-------------------|----------------------|--------------------------------------------------------------------------------|
| PXE server IP | 172.16.9.1 | `playbook/pxe_server_setup.yml` (dnsmasq config, iPXE script, samba conf, webapp env), `playbook/startnet.cmd` (mount paths), `boot-tools/blancco/grub-blancco.cfg` (TFTP/HTTP URLs) |
| PXE subnet | 172.16.9.0/24 | Same as above, plus `playbook/pxe_server_setup.yml` (UFW rules) |
| DHCP range | 172.16.9.10-100 | `playbook/pxe_server_setup.yml` (dnsmasq config) |
| DHCP range | 172.16.9.10-100 | `playbook/site-vars.yml` (`pxe_dhcp_start`, `pxe_dhcp_end`) |
| NIC MACs / names | (required) / pxe0, corp0 | `playbook/site-vars.yml` (`pxe_nic_mac`, `corp_nic_mac`, `pxe_nic_name`, `corp_nic_name`) |
| Hostname | pxeserver | `autoinstall/user-data` (identity.hostname) |
### Identity and credentials

View File

@@ -25,8 +25,11 @@ The NIC guard catches those.
| `/run/pxe-nic-status.json` | last result, read by the webapp |
Sources: `playbook/pxe-server-helpers/`, installed by `pxe_server_setup.yml`
(vars `pxe_imaging_mac`, `pxe_ge_mac`, `pxe_corp_domain_suffixes`,
`pxe_nic_allowed_switches`).
(vars `pxe_nic_mac`, `corp_nic_mac`, `pxe_corp_domain_suffixes`,
`pxe_nic_allowed_switches`; `pxe_imaging_mac` / `pxe_ge_mac` are derived
from the first two). The same MACs pin the NICs in netplan
(`playbook/templates/netplan-pxe.yaml.j2`, names `pxe0` / `corp0`) and in
`/etc/dnsmasq.d/pxe-nic-only.conf`.
## How it decides