pxe-server/docs/pxe-nic-guard.md

PXE NIC guard

The PXE server's DHCP/PXE (dnsmasq) must never serve on the GE corporate network. Each server has two NICs:

Role	Network	Address
imaging	isolated imaging switch	static 172.16.9.1/24, dnsmasq
ge	GE corporate network	DHCP client (systemd-networkd)

/etc/dnsmasq.d/pxe-nic-only.conf pins dnsmasq to the imaging NIC by name. That stops dnsmasq answering on the GE NIC, but not on the imaging NIC when that NIC itself ends up on the wrong network: a cable swap, an imaging switch uplinked into a GE port, or a second PXE server on the same switch. The NIC guard catches those.

What is installed

File	Purpose
/usr/local/sbin/pxe-nic-check	the classifier
/etc/pxe-server/nic.conf	which NIC is which
/etc/systemd/system/dnsmasq.service.d/nic-guard.conf	guard before every dnsmasq start
/etc/systemd/system/pxe-nic-check.{service,timer}	watch every 5 min
/usr/local/sbin/pxe-nic-carrier	re-check on imaging cable moves
/etc/networkd-dispatcher/<state>.d/50-pxe-nic-carrier	calls it on link changes
/etc/systemd/system/pxe-nic-carrier.service	the re-check after carrier gain
/run/pxe-nic-status.json	last result, read by the webapp

Sources: playbook/pxe-server-helpers/, installed by pxe_server_setup.yml (vars pxe_nic_mac, corp_nic_mac, pxe_corp_domain_suffixes, pxe_nic_allowed_switches; pxe_imaging_mac / pxe_ge_mac are derived from the first two). The same MACs pin the NICs in netplan (playbook/templates/netplan-pxe.yaml.j2, names pxe0 / corp0) and in /etc/dnsmasq.d/pxe-nic-only.conf. With pxe_nic_mac empty the playbook picks the imaging NIC at install time with pxe-pick-nic --auto, which runs this check on the one port with a link (--iface X --probe, 65 s) and accepts only "isolated". With corp_nic_mac empty ge_mac is blank and the GE NIC is whichever other port holds a DHCP lease.

How it decides

It listens on each NIC (raw socket, sends nothing; it joins the STP, 802.1X, LLDP, CDP and PVST+ multicast groups so a NIC that is not promiscuous still hears them) for:

• LLDP and CDP: managed switches announce themselves. GE switches do; the unmanaged imaging switch does not.
• 802.1X EAPOL from an authenticator: GE ports ask for port authentication (EAP Request, Success or Failure). EAPOL that clients send is ignored: iPXE sends three EAPOL-Starts every time it opens a NIC and Windows sends Starts and EAP Responses, so a bay PXE-booting on the imaging switch is not mistaken for a GE port. Those frames show up only as "N client EAPOL frames ignored".
• DHCP OFFER/ACK from any other server.
• STP and the count of distinct ARP hosts: reported, never decisive.

On the imaging NIC it also sends one DHCPDISCOVER and collects offers for 6 s. On the GE NIC it reads the networkd lease instead (no traffic).

Verdict	Meaning
isolated	no foreign DHCP, no LLDP/CDP/EAPOL
ge	LLDP, CDP or EAPOL seen, or a DHCP server whose domain ends in a corp suffix (geaerospace.net, ge.com; whole labels, so notgeaerospace.net does not count)
foreign-dhcp	another DHCP server, no corp domain
no-link	no carrier
error	the check itself failed

Healthy: imaging NIC isolated (or no-link), GE NIC ge.

Our own dnsmasq never counts. An offer whose server id is our own address but comes from a MAC that is not ours is a second PXE server and DOES count.

If the imaging switch is itself a managed switch that sends LLDP/CDP, put its LLDP system name, CDP device id or chassis MAC in allowed_switches (pxe_nic_allowed_switches), or the guard will block dnsmasq. 802.1X is never allowed.

Modes

• pxe-nic-check - check every NIC (60 s), print a summary. --json, --write, --probe (imaging DHCP probe), --iface NAME, --window S.
• pxe-nic-check --guard - dnsmasq ExecStartPre. Imaging NIC only, about 6 s. Exit 1 = do not serve: dnsmasq does not start. A bug or no link exits 0 (fail open) with an error line in the journal.
• pxe-nic-check --watch - the timer. On a bad imaging verdict it runs systemctl stop dnsmasq and logs CRIT. It never starts dnsmasq. If dnsmasq was (re)started during the 60 s window (an operator, or the carrier hook, restarted it and its guard passed), the watch re-checks for about 6 s and stops dnsmasq only if that still says blocked; if dnsmasq is already stopped it only logs.
• pxe-nic-carrier - the networkd-dispatcher hook. Only the imaging NIC (or its bridge ports) counts; GE NIC events are ignored. Carrier lost: dnsmasq is stopped at once if it was running. Carrier back: after 2 s, pxe-nic-carrier.service restarts dnsmasq if it was running or the hook stopped it, so the guard checks the new network before dnsmasq serves again; dnsmasq that an operator or the watch stopped stays stopped. A bridge with several ports stops dnsmasq when any one of them loses carrier. Log: journalctl -t pxe-nic-carrier.

Every start or restart of dnsmasq runs the guard, so systemctl restart dnsmasq and the @reboot cron restart take about 6 s longer than before.

Reading it

sudo pxe-nic-check                       # full summary, 60 s
cat /run/pxe-nic-status.json             # last guard/watch result
journalctl -u dnsmasq -b | grep pxe-nic-check
journalctl -u pxe-nic-check -b
journalctl -t pxe-nic-carrier -b         # cable unplug / replug

dnsmasq will not start

1. journalctl -u dnsmasq -b | grep pxe-nic-check shows the evidence, e.g. REFUSING to start dnsmasq: imaging NIC br-pxe verdict ge: LLDP from WJ-IDF2 port Gi1/0/14.
2. Check the imaging cabling: the imaging NIC must go to the imaging switch only, and that switch must not be uplinked anywhere.
3. sudo pxe-nic-check until the imaging NIC reads isolated.
4. sudo systemctl start dnsmasq (runs the guard again).

Floor tests (run with the floor idle)

All three tests stop imaging for a few minutes. Have console or GE-side SSH access to the server; do not rely on the imaging network for the session.

(a) Rogue DHCP on the imaging switch: expect the alarm

1. Confirm the baseline: sudo pxe-nic-check --probe shows the imaging NIC isolated and dnsmasq_bound true.
2. Plug a device that runs a DHCP server into a free port of the IMAGING switch (a spare home router with DHCP on, LAN port only, nothing in its WAN port; or a laptop running dnsmasq).
3. Run the watch now instead of waiting up to 5 min: sudo systemctl start pxe-nic-check.service
4. Expect:
   • journalctl -u pxe-nic-check -n 20 has a CRIT line imaging NIC is NOT isolated, stopping dnsmasq: ... verdict foreign-dhcp: DHCP from 192.168.x.1, ... (or ge if the router sends LLDP).
   • systemctl is-active dnsmasq is inactive.
   • /run/pxe-nic-status.json has "dhcp_guard": {"state": "blocked", ...}.
5. Prove the guard too: sudo systemctl start dnsmasq must FAIL, and journalctl -u dnsmasq -n 20 shows REFUSING to start dnsmasq.
6. Unplug the rogue device. sudo pxe-nic-check --probe shows isolated. sudo systemctl start dnsmasq starts. PXE-boot one client to confirm.

(b) Cable swap into a GE port: expect the guard to refuse dnsmasq

dnsmasq must be stopped BEFORE the cable moves, so it never serves on GE.

1. sudo systemctl stop dnsmasq; systemctl is-active dnsmasq must say inactive. Do not reboot during the test (the @reboot cron restarts dnsmasq; the guard would still block it, but do not test that way).
2. Move the IMAGING NIC's cable from the imaging switch into a live GE network port. Wait for link: ip -br link shows the imaging NIC UP. The carrier hook sees the link come up and runs the guard once (one DHCPDISCOVER) to refresh the status file; with dnsmasq stopped it does not start it. journalctl -t pxe-nic-carrier -n 5 shows dnsmasq was not running, left stopped.
3. Run the guard by hand first: sudo pxe-nic-check --guard; echo "exit $?" Expect exit 1 and a REFUSING to start dnsmasq line naming LLDP/CDP, 802.1X, or DHCP from 10.x.x.x, domain ....geaerospace.net. This sends one DHCPDISCOVER on the GE port, the same as any PC being plugged in. To send nothing, use --no-probe --window 60 instead and rely on LLDP/CDP/802.1X (switches announce every 30-60 s). If it exits 0, STOP: do not start dnsmasq. Move the cable back and send the output of sudo pxe-nic-check --probe --json.
4. Only after step 3 exited 1: sudo systemctl start dnsmasq. Expect Job for dnsmasq.service failed; systemctl is-active dnsmasq says failed; journalctl -u dnsmasq -n 20 shows REFUSING.
5. Move the cable back to the imaging switch. sudo pxe-nic-check --probe shows the imaging NIC isolated. sudo systemctl start dnsmasq starts. PXE-boot one client to confirm.

(c) Imaging cable unplugged and replugged: expect stop, re-check, start

1. dnsmasq running, imaging NIC isolated.
2. Unplug the imaging NIC's cable. Within a second journalctl -t pxe-nic-carrier -n 5 shows carrier lost on imaging NIC ... stopped dnsmasq and systemctl is-active dnsmasq is inactive.
3. Plug it back into the imaging switch. About 10 s later the journal shows guard passed, dnsmasq (re)started and dnsmasq is active. Had it gone into a GE port instead, the guard refuses: the journal shows dnsmasq NOT started and dnsmasq stays down.

Status file

/run/pxe-nic-status.json, mode 0644, replaced atomically:

{"checked_at": "2026-10-14T16:47:09-04:00", "duration_s": 60,
 "dhcp_guard": {"state": "allowed"|"blocked"|"unknown", "reason": "..."},
 "interfaces": [{"iface": "br-pxe", "mac": "aa:bb:..", "role": "imaging"|"ge"|"other",
   "link": "up"|"down", "speed_mbps": 5000, "addrs": ["172.16.9.1/24"],
   "verdict": "isolated"|"ge"|"foreign-dhcp"|"no-link"|"error", "evidence": ["..."],
   "dhcp_servers": [{"server": "10.1.2.3", "domain": "x.geaerospace.net", "own": false}],
   "switch": {"lldp": [{"system": "...", "port": "...", "vlan": null}],
              "cdp": [{"device": "...", "port": "..."}], "eapol": false, "stp": false},
   "arp_hosts": 3, "dnsmasq_bound": true}]}

dnsmasq_bound is null when it cannot be told (no ss). A bridge (br-pxe) is reported as one interface; its ports are listened on too, because LLDP and 802.1X frames stop at the port and never reach the bridge.