Commit Graph

501 Commits

Author SHA1 Message Date
cproudlock
d63cc5761b Image import keeps the shared BPRT link and seeds BPRT on a fresh box
An MCL import merged Deploy/Applications with rsync -a, which replaced each
image's BPRT symlink with a plain copy of the vendor folder. That copy has
no site overlay, so standard/engineer enrollment lost
Register-BulkEnrollTask-Standard.ps1 (specialize Order 39).

Now Deploy/Applications is imported first and without BPRT:
- where _shared/Applications/BPRT exists it is left alone (the BPRT page
  keeps it current) and the image link is put back;
- on a box with none yet, the imported BPRT seeds it: vendor
  Select-PCConfig.ps1 kept as -vendor, our overlay added, BOM fix applied;
- the import refuses before anything moves when that folder would not work
  (vendor scripts missing, or no overlay copy to add).

The two Register scripts join Select-PCConfig.ps1 in the repo. The playbook
already syncs shopfloor-setup to the enrollment share, so
config.BPRT_OVERLAY_DIR finds all three there; overlay_sources now falls
back to it for every overlay file, not only the shim.
2026-10-16 14:56:33 -04:00
cproudlock
821a927365 Engineering Layer: survive the C:\Deploy cleanup; the employee gets the icon
Engineer builds left the employee without "Configure my Engineering
Layer". First logon ran C:\Deploy\Applications\extra_engineering\
Install-EngineeringLayer.ps1, and the script itself lived in C:\Deploy:
std/eng have no autologon, so a PC that rebooted before the employee
signed in had lost C:\Deploy to the orchestrator's DeleteDeployDir task,
and nothing installed, silently.

- engineer unattend, specialize Order 37 (free slot): run the script with
  -StageOnly, copying the payload and the script to C:\PreInstall\engineering
  while C:\Deploy exists (Path 119 chars)
- first logon Order 11: run it from C:\PreInstall\engineering, falling back
  to C:\Deploy if staging did not happen
- the script (now in the repo, playbook/engineering/; it lived only on the
  server): after the install, restore the Public desktop icon if it is
  missing and the PC is not configured yet (no C:\TMP\DUU_Complete.log), and
  register Active Setup so each later user gets the icon on their own
  desktop when the Public one is gone. v3.1 puts the icon on the Public
  desktop; clicking it (configure, from GE's V: share) deletes it for all
- playbook deploys the script to gea-engineer extra_engineering; the 650 MB
  payload stays site content (documented in SITE-CUSTOMIZATION.md)

Tested in the win11 VM with the exact unattend strings and a stand-in
installer that skips the icon: specialize staged payload and script;
C:\Deploy deleted; first logon installed from the staged copy, restored the
icon and registered Active Setup. The per-user step as a new user: icon on
their desktop when the PC is unconfigured and the Public icon is gone; none
when configured or when the Public icon is there. Lint clean. Not deployed:
unattends go out only through scripts/deploy-unattend.sh.
2026-10-16 12:37:50 -04:00
cproudlock
9d75d03b2d Standard/Engineer builds are done at OOBE on the imaging floor
Everything after OOBE needs the employee to log in, which happens off the
PXE network, so the floor waited forever at step 3 of 5. Now 3 steps:
WinPE, Windows setup installing apps, ready at OOBE (succeeded).

- both unattends: specialize Order 49 posts step 3 as succeeded
  (Path 182 chars; lint clean). Only that line changed.
- Send-ImagingStage.ps1: StageTotal 3; steps above the total (the old
  first-logon steps 4 and 5 still in the unattends) are logged and not
  posted, so they cannot reopen a finished build
- winpe-status-push.ps1: total 3 for gea-standard / gea-engineer
- floor tile step 3: "Ready at OOBE"

Tested in the win11 VM against the test server: steps 2 and 3 post,
step 3 shows succeeded, step 4 is skipped. Not deployed: the unattends
go out only through scripts/deploy-unattend.sh.
2026-10-16 10:09:40 -04:00
cproudlock
bca04f4ca2 Verified MCL uploads: hash every file, repair over HTTP, gate the import
Small Control JSONs have arrived empty or wrong more than once, and the old
check compared sizes only and skipped missing files. A copy of
Deploy/Control on the dev PC still holds two 0-byte JSONs.

Upload-Image.ps1 (tested on Windows PowerShell 5.1 in the win11 VM):
- refuses an unfinished MCL cache: exactly one <OS>-Production.tag (MCL
  writes it last), no Control file newer than it, selected OS matches,
  every Control/Tools JSON non-empty and valid
- checks payload against the SHA-256 MCL's own Control JSONs publish. MCL
  resumes downloads onto old files (FileMode.Append) and then trusts any
  file of the right size: the September junk-tail corruption. Offers to
  delete the bad cache files so MCL downloads them again
- hashes every file into a manifest, robocopies, POSTs the manifest
- re-sends bad small files over HTTP (no proxy), large ones by deleting
  the bad copy and robocopy /J (robocopy skips a same-size, same-time file
  as "modified" even with /IS /IT)

Webapp:
- services/upload_verify.py hashes the upload on the server's own disk in
  a background thread; reports missing, differing, bad-in-MCL-cache and
  stray files; PUT /api/upload/file writes temp, checks hash, renames
- import of the upload folder only after a passed verification, with
  every file unchanged (size, mtime, ctime, inode), no stray files, and
  the target the OS allows (LTSC: Shop Floor only; else Standard or
  Engineer; unknown OS: none). Import, verify and PUT share one lock
- paths matched case-insensitively against the disk, as Samba does
- import page shows the upload, MCL run, verification and stray files
  with a remove button; record of each import kept on the image

Reviewed adversarially; all 11 confirmed findings fixed with tests.
42 new unit tests. VM test: clean upload; same-size same-mtime damage on
3 files repaired in one round; stray file blocks import until removed;
Shop Floor refused for a Pro upload; import records and clears the
folder; missing tag, 0-byte JSON, junk-head WIM and late Control file
each refused with nothing uploaded.
2026-10-16 10:09:40 -04:00
cproudlock
576955785d Faster shopfloor-setup sync, prompt spacing, XP-era test client
- Deploy shopfloor-setup with rsync instead of the copy module: the
  ~4 GB tree took about 4 minutes on every re-run because copy checksums
  each file. rsync transfers only changes; changed only when it did.
- iPXE drops a trailing space, so 'Machine number:' ran into the typed
  digits. The space now comes from a hex 20 setting.
- tests/vm: pxetest-xp is a permanent client (SeaBIOS, qemu32 32-bit CPU,
  rtl8139, 1 GiB, IDE disk) for the i686 Clonezilla path; start.sh xp.
  Only one client may run at a time across all three.
2026-10-15 21:43:33 -04:00
cproudlock
23ee99a390 Clonezilla: warn while the default share password is in use; prepare-boot-tools.sh no longer fails on an empty tool folder
The final listing of an empty boot-tools/blancco (no Blancco ISO yet)
failed under pipefail and made the script exit 1 after a good download.
2026-10-15 15:32:10 -04:00
cproudlock
9d56a5369e Clonezilla back up / restore by machine number
Boot menu: "Clonezilla: back up this PC" and "restore this PC" ask for the
machine number and boot Clonezilla (64-bit with Debian shim, or 32-bit
i686 by cpuid) with ocs_repository=smb3://clonezilla:<pw>@server/clonezilla/,
ocs_prerun fetching clonezilla-scripts/pxe-cz.sh and ocs_live_run running
it. The script validates the machine number, saves
<machine>-<YYYY-MM-DD-HHMM> (server time), writes pxe-meta.txt, runs
ocs-chkimg, and restores only that machine's images with -k1 -r after a
typed YES. The old entry is "Clonezilla: manual".

Playbook: clonezilla Samba user for the [clonezilla] share only, password
from clonezilla_smb_password (validated, filled into the menu
placeholder), scripts installed to the web root.

Webapp: Clonezilla backups page lists image folders and zips grouped by
machine number with serial/model/version/check, serial mismatch flag,
delete, move to another machine, streamed tar download, disk space bar
and an explicit keep-last-N retention with preview. Tests and a VM test
plan added.
2026-10-15 15:26:30 -04:00
cproudlock
dde1aacccb Clonezilla for XP-era PCs: 32-bit build picked by CPU
Current Clonezilla is amd64-only. The menu now checks the CPU with iPXE's
cpuid --ext 29 (long mode): 64-bit CPUs get the current release (through
Debian's shim under Secure Boot), 32-bit-only CPUs (Pentium 4 without
EM64T, Pentium M, early Atom) get Clonezilla 3.2.0-5 i686, the last
32-bit release, from clonezilla-i686/. Without a cpuid command the 32-bit
build is used, which also runs on 64-bit CPUs; UEFI always gets the
64-bit build. A 'Clonezilla Live 32-bit' item forces it.
prepare-boot-tools.sh downloads it, the playbook deploys it.
2026-10-15 14:50:40 -04:00
cproudlock
54015566a7 Menu: Clonezilla boots with Secure Boot on (Debian shim); Memtest86+ removed
Clonezilla Live runs a Debian-signed kernel whose modules (usb-storage,
uas, exfat) are signed by the same build key, so the menu now starts it
through Debian's Microsoft-signed shim with the iPXE shim command, the
same way Blancco uses Ubuntu's. prepare-boot-tools.sh extracts that shim
from the Clonezilla release (EFI/boot/bootx64.efi -> clonezilla/
shimx64.efi) and warns when the shim is not Microsoft-signed or the
kernel not Debian-signed.

Memtest86+ is unsigned and needed Secure Boot off; memory tests now use
the Dell built-in diagnostics (F12 at power-on, Diagnostics), noted in
the menu. The playbook removes the old memtest web dir. Nothing on the
menu needs Secure Boot off any more; the 'disable Secure Boot' screen
remains only as the fallback when a signed path fails.
2026-10-15 14:47:25 -04:00
cproudlock
c8e0b4eace docs: plain ASCII in the new build-usb example 2026-10-15 14:39:21 -04:00
cproudlock
da6e62c084 Merge branch 'fix-build' into pxe-webapp-redesign 2026-10-15 14:38:48 -04:00
cproudlock
0b1625ed87 Build/playbook re-run safety: corp access, netplan rollback, first-boot retry
H1  corp_admin_cidrs (UFW 22/9009 on the corp NIC from these only; refuses
    when it would lock out the current SSH client) and corp_ssh_password_auth
    (sshd drop-in, passwords only from the imaging subnet). Defaults keep the
    current behaviour and print a loud warning.
H2  pxe-netplan-apply.sh: pre-flight refusal (pxe-MAC NIC holds the default
    route, a DHCP lease or the route to an SSH client), run from the USB
    before anything changes; post-apply checks (names, imaging IPv4, corp
    IPv4 back within 30 s, SSH route) and a 120 s systemd-run auto-revert.
H3  restore() puts the previous NIC names back (two-phase rename via
    temporary names, so swaps work) before generate + apply.
H4  a failed rename is fatal (restore + exit 1); the play drops the old
    any-interface UFW rules only after verifying both NICs and their
    per-interface SSH rules.
H5  first-boot checks the playbook exit code, retries at next boot (max 5),
    writes /var/lib/pxe-firstboot.failed and /etc/issue + /etc/motd banners.
M1  build-usb.sh verifies iPXE files against the playbook pins, every wheel
    against its own package's hashes, drops the unpinned pip fallback,
    requires --site-vars or --no-site-vars.
M2  webapp restarts when the code, wheels or unit changed.
M3  ignore-carrier on the imaging NIC; ip_nonlocal_bind kept for the
    apache/networkd boot race.
M4  menu suffix must match blancco_kver; missing kernel with no earlier
    build is fatal; a live menu with the staged Blancco test entry is not
    overwritten.
M5  build-proxmox-iso.sh is unsupported (hard exit); use build-usb --image.
M6  deploy-unattend.sh: password via SSHPASS env and sudo stdin only,
    %q-quoted remote args, sha256 check of the live file; lint-unattend.py
    reads PXE_SERVER/PXE_USER/PXE_PASS and uses sshpass -e.
M7  the playbook lints the unattend files before the force: yes copies
    (lint-unattend.py ships on CIDATA as scripts/); pre-commit ACMR.
M8  Blancco initramfs build: no ignore_errors, pipefail, trap cleanup.
Also: Send-ImagingStage.ps1 deploy, carrier hook install tasks
(pxe-nic-carrier, networkd-dispatcher), L1 imaging-active hold
(allow_disruptive) and no iPXE downgrade below an Updates-page install,
L2 iso-volid, L3 kver pin read from the playbook, L4 cp -rL, L5 refuse
>= 4 GiB WinPE files, L6 PIP_DIR wipe guard, L7 per-package wheel hashes,
no online fallback, L8 check-updates.py (apt_pkg/cached compare, all
components, exit 2 on no debs / bad answers).
2026-10-15 14:32:23 -04:00
cproudlock
b20e6be629 Merge branch 'fix-boot' into pxe-webapp-redesign 2026-10-15 14:27:35 -04:00
cproudlock
21ff95af78 NIC guard: ignore client EAPOL, re-check on carrier change, SMB-only lease cleanup
- pxe-nic-check: count 802.1X only from an authenticator (EAPOL type 0
  with EAP Request/Success/Failure). iPXE sends EAPOL-Start on every NIC
  open and Windows sends Start/Response, so a PXE-booting bay made the
  verdict 'ge' and the watch stopped dnsmasq for the floor.
- pxe-nic-check: join the STP/PAE/LLDP/CDP/PVST+ multicast groups on
  every capture socket, so CDP-only ports are heard on a plain NIC.
- pxe-nic-check --watch: if dnsmasq was (re)started inside the window,
  re-check (~6 s) before stopping it; skip when already stopped.
  New --state-cmd and --print-imaging.
- pxe-nic-carrier (+ pxe-nic-carrier.service): networkd-dispatcher hook
  for the imaging NIC only. Carrier loss stops dnsmasq; carrier gain
  restarts it through its guard if it was running or the hook stopped it.
- pxe-dhcp-hook.sh: clear only SMB (tcp 445/139) conntrack entries and
  sockets, not all traffic for the client, which raced iPXE's first
  HTTP fetch.
- GetPxeScript.aspx: every boot line has a failure branch (iPXE 2.0
  accepts an unsigned EFI kernel and fails at boot under Secure Boot,
  which aborted the script).
- Tests: client EAPOL, packet_mreq packing, watch race, shell helpers on
  stub PATH, lab scenarios eapol-client, br-eapol-client, race, carrier.
2026-10-15 14:26:39 -04:00
cproudlock
606659bf07 webapp: fix review findings (imaging types, NIC status, iPXE gates)
- imaging: coerce client payload and stored session fields to str/int,
  drop lists/dicts; tiles template guards pctype/current_stage; POST
  rejects non-object bodies and non-string serials.
- nic_status: normalise every check-file entry; load nic.conf as-is with
  a [nic] fallback; split interface= lists; keep the check's binding while
  fresh, resolve wildcard sockets from the dnsmasq pin (never True for an
  excluded NIC); mark vanished NICs down; include role bridges; stale or
  future-dated files downgrade to warnings and drop evidence.
- Overview renders the network pill in its own try and shows
  'Network: unknown' when status fails.
- ipxe_update: read every Authenticode signature (all WIN_CERTIFICATE
  entries plus nested SPC_NESTED_SIGNATURE) with openssl; shim swap and
  install need both Microsoft UEFI CA 2011 and 2023; install re-reads
  each file (snponly ipxe.efi only); staged.json schema version, older
  records shown stale and refused; uhci counts as native USB.
- backups describe their files; risky rollbacks need a confirm flag.
- mac_folders: guarded listdir, cached describe, configured TFTP root,
  warning for a shim without ipxe.efi.
- evidence pack: years 2000..2100, 400 days inclusive.
- a11y role=group on quick ranges; progress bar stays full during 'shim'.
- tests/test_review_fixes.py: 29 unittest cases incl. both real shims.
2026-10-15 14:20:24 -04:00
cproudlock
53d39bfc50 tests/vm: Blancco RAM note, menu keys 2026-10-15 11:39:01 -04:00
cproudlock
24c41d7e87 build-usb: count PPKGs/wheels/debs through symlinked payload dirs 2026-10-15 11:04:41 -04:00
cproudlock
2717439690 Merge branch 'pxe-webapp-redesign' into usb-build 2026-10-15 10:59:30 -04:00
cproudlock
bcd91b488a Offline install that lands fully patched: resolve against the ISO's own base
- autoinstall: dead mirror + fallback offline-install, so the installed
  base is exactly the ISO's; late-commands run offline-apt-install.sh on
  CIDATA packages/ (local apt repo, one transaction, errors fail the
  install, log /var/log/offline-apt-install.log); first boot no longer
  runs dpkg -i; real archive sources written back at the end.
- download-packages.sh --iso: unpacks the ISO's default install source
  into a docker image (pxe-iso-base:<version>) and downloads there: all
  updates of the base, the explicit list (zip included) and the newest
  pool packages with their closure; writes Packages + explicit.list and
  checks the set offline in a fresh base container (--network none).
- offline-apt-install.sh: new; also used by that check (--simulate).
- build-usb.sh: ships Packages, explicit.list and the script; refuses a
  set without them.
- tests/vm: pxetest-client-nosb (Secure Boot off, e1000e, SATA), one
  client at a time.
2026-10-15 10:59:30 -04:00
cproudlock
bab6927bdb prepare-boot-tools.sh: current Clonezilla (3.3.3-37) and Memtest86+ (8.10)
Clonezilla 3.2.1-6 is gone from the mirrors (the download failed and the
unzip aborted the script before Memtest). Memtest moved its downloads to
download/v<ver>/, and 8.x names the x86_64 UEFI binary mt86p_<ver>_x86_64
with no .efi suffix; the script now finds it.
2026-10-15 10:42:56 -04:00
cproudlock
c1a5fb3e02 Merge branch 'pxe-webapp-redesign' into usb-build 2026-10-15 09:57:15 -04:00
cproudlock
86b4c1f301 tests/vm: quiet ISO download progress 2026-10-15 09:52:08 -04:00
cproudlock
18f20b00e2 Offline bundle: pin and hash the webapp wheels, refuse unpinned ones, check for updates
- webapp/requirements.txt pins all eight wheels with sha256, at the fixed
  releases (lxml 6.1.3 for GHSA-vfmq-68hx-4jfw / PYSEC-2026-87, werkzeug
  3.1.9 for GHSA-g6x2-hccm-hh4m).
- download-packages.sh fetches exactly that set for Python 3.12 / x86_64
  into an emptied folder; PIP_DIR is honoured inside the container (own
  mount), files are handed back to the calling user, USE_DOCKER=1 forces
  the container on a noble host, an unknown package name now prints apt's
  error instead of failing silently, and linux-modules-extra-7.0.* is gone
  (it does not exist; 7.0 ships those modules in linux-modules).
- Playbook: the wheel installer refuses any wheel whose sha256 is not
  pinned, needs exactly one wheel per pin, and removes an older installed
  version by its RECORD first. The webapp copy is a full rsync every run
  (was cp with creates: app.py, which never updated an install).
- scripts/check-updates.py: offline debs against noble security/updates
  and the pins against PyPI + osv.dev; exit 1 when something needs
  updating. Run it before building a USB.

Tested: installer in a Python 3.12 venv (clean install, upgrade over lxml
6.0.4 + werkzeug 3.1.8 leaves no old dist-info, re-run is a no-op, a stray
wheel is refused); check-updates.py flags the April bundle (94 security
updates) and passes the pins.
2026-10-15 09:41:16 -04:00
cproudlock
6dc9761e67 build-usb: --site-vars puts the per-site vars on CIDATA; harness pins the VM NIC MACs
tests/vm/build-image.sh writes site-vars.yml with pxe_nic_mac (isolated
network NIC) and corp_nic_mac (NAT NIC) and passes it to build-usb.sh.
2026-10-15 08:56:45 -04:00
cproudlock
29d815fbeb Merge branch 'pxe-webapp-redesign' into usb-build 2026-10-15 08:55:40 -04:00
cproudlock
45cf7be37a build-usb: whole webapp, hard sfdisk failure, ISO-derived boot params, image-file mode; tests/vm harness
- Copy the whole webapp/ tree (minus __pycache__) in build-usb.sh and
  build-proxmox-iso.sh, so config, services/, views/ and
  blancco_evidence_monthly.py reach the server.
- sfdisk --append failure now aborts; the CIDATA partition is the one
  starting exactly at the computed sector, and nothing that starts
  before it is ever formatted (the old fallback hit the ISO's EFI
  partition).
- Volume id and El Torito load sizes come from the ISO
  (xorriso -report_el_torito as_mkisofs); grub.cfg must contain
  autoinstall after patching.
- --image FILE SIZE|auto builds the same layout in a file without root
  (mkfs.vfat -C + mtools, sfdisk on the file). Plus --payload-root,
  --skip-enrollment/-bios/-drivers/-boot-tools, --yes.
- tests/vm: libvirt harness (pxetest-server, pxetest-client, isolated
  pxetest-imaging network) with fetch-iso/build-image/create/start/
  status/stop/destroy.
2026-10-15 08:55:40 -04:00
cproudlock
ead3314f58 Two-NIC standard: pin imaging and GE NICs by MAC everywhere
Every server has an imaging NIC (static, the only one that serves
DHCP/TFTP/HTTP boot/SMB) and a GE corporate NIC (DHCP client, SSH and the
webapp on 9009). The playbook guessed the PXE NIC and the installer put
172.16.9.1/24 on every en* NIC, so a two-NIC box could answer on the GE
network.

- New per-site vars pxe_nic_mac / corp_nic_mac (required, from
  playbook/site-vars.yml or -e; template site-vars.example.yml), stable
  names pxe_nic_name / corp_nic_name (default pxe0 / corp0, overridable),
  pxe_server_ip, pxe_subnet, DHCP range. pxe_imaging_mac / pxe_ge_mac are
  derived from them. The play stops before touching anything if a MAC is
  unset, malformed, duplicated or not on the box (lists the box's NICs).
- Heuristic NIC detection and the single-NIC netplan task replaced by a
  two-NIC netplan template (match macaddress + set-name), installed by
  pxe-netplan-apply.sh: validate in a scratch root, back up, generate,
  rename, apply, restore the previous config on any failure. cloud-init
  network config disabled so it cannot rewrite it.
- dnsmasq: /etc/dnsmasq.d/pxe-nic-only.conf from the vars (interface=,
  except-interface=, no-dhcp-interface=); dnsmasq.conf IP/range from vars.
- UFW per interface: imaging NIC 22, 67/udp, 69/udp, 80, 4433, 445, 139,
  137/udp, 138/udp, 9009; GE NIC 22 and 9009 only. Old any-interface rules
  and the stale 9011 rule removed after the new ones are in.
- sysctl drop-in: IPv4/IPv6 forwarding off; ip_nonlocal_bind so Apache
  can bind the imaging IP before the address is up.
- Samba [global]: interfaces = 127.0.0.0/8 <imaging NIC>, bind interfaces
  only. Apache 80 and 4433 on the imaging IP only; 9009 stays on all (TODO:
  the webapp has no auth).
- autoinstall: DHCP only on en*, no static 172.16.9.1; apt geoip off and
  offline-install fallback now that the corp NIC may have a lease.
- Blancco UEFI/Secure Boot: blancco_kver (7.0.0-34-generic) drives the
  kernel, initramfs and modules builds instead of the running kernel;
  outputs vmlinuz-ubuntu-70, kexec-initrd-70.img, kmod-<kver>.tar.gz as the
  menu and blancco-init.sh expect; creates: guards keyed on the full kver.
  Ubuntu's shimx64.efi.signed published as blancco/shimx64.efi.
  download-packages.sh adds shim-signed, linux-generic-hwe-24.04 and the
  pinned linux-image/modules/modules-extra for BLANCCO_KVER.

Rendered with the live MACs and names, netplan generate output is
identical to the live config; pxe-nic-only.conf and the Samba binding
match live directive for directive.
2026-10-15 08:53:41 -04:00
cproudlock
689eebf9d1 Merge branch 'nic-guard' into pxe-webapp-redesign 2026-10-15 08:40:57 -04:00
cproudlock
d3a0f90ca1 Re-imaging the same PC: clear its stale SMB/conntrack state every time
A PC whose imaging failed and is re-imaged gets the same IP, and the
server still held state from the dead run, so net use failed (System
error 53). The April fix (18537ac) did not cover it:

- The dnsmasq hook only acted on 'add'/'del'. A re-image comes back to the
  existing lease, which dnsmasq reports as 'old', and only when the
  hostname/MAC changed. Now dnsmasq sets script-on-renewal, and the hook
  clears state on 'old' when the request comes from PXE firmware or iPXE
  (vendor class PXEClient), i.e. the machine just network-booted. A
  Windows lease renewal (MSFT 5.0) is left alone.
- The 1 h conntrack timeout never applied after a reboot: systemd-sysctl
  runs before ufw loads nf_conntrack, so the keys did not exist and the
  5-day default stayed. /etc/modules-load.d/pxe-conntrack.conf loads it
  first.
- smb-soft-reset.sh flushed neighbours on br-pxe, which no longer exists.

Live 2026-10-14. Hook tested on the server with an unused IP: network-boot
renewal clears, Windows renewal and no vendor class do not, add clears.
2026-10-14 17:19:33 -04:00
cproudlock
bc422649e9 NIC guard: dnsmasq never serves DHCP/PXE on the GE network
pxe-nic-check classifies each NIC from what it hears: LLDP/CDP/802.1X
from managed switches, foreign DHCP offers and their domain (one
DHCPDISCOVER on the imaging NIC; the GE NIC's networkd lease is read
instead), STP and ARP hosts for information. A kernel BPF filter keeps
bulk SMB traffic out of the capture loop; bridge ports are listened on
too, since LLDP and EAPOL stop at the port.

- dnsmasq drop-in runs --guard as ExecStartPre: exit 1 on a ge or
  foreign-dhcp imaging verdict, fail open (loudly) on bugs or no link
- pxe-nic-check.timer runs --watch every 5 min: stops dnsmasq on a bad
  imaging verdict, never starts it
- /run/pxe-nic-status.json (0644, atomic) for the webapp
- playbook installs it all and writes /etc/pxe-server/nic.conf from
  pxe_imaging_mac, pxe_ge_mac, pxe_corp_domain_suffixes,
  pxe_nic_allowed_switches
- unit tests plus a no-root network-namespace lab (veth pairs, a raw
  socket DHCP/LLDP peer and real dnsmasq); docs/pxe-nic-guard.md has the
  operator notes and the two floor tests
2026-10-14 17:18:53 -04:00
cproudlock
28fd9fe474 Fix the unattend that broke Engineer builds; gate unattend deploys on the linter
a94c203 added a RunSynchronous Path of 417 chars to both std/eng answer
files. Windows caps it at 259 and then rejects the WHOLE answer file, so
an Engineer build stopped at 'Windows could not parse or process unattend
answer file'. The live files were restored to their pre-a94c203 versions
the same day.

- The status steps now fit: Send-ImagingStage.ps1 -Persist copies itself
  out of C:\Deploy (no cmd copy chain), short PowerShell switches, default
  arguments left out. Longest Path is 168 chars; lint-unattend passes.
  NOT deployed: these files must go live only through deploy-unattend.sh.
- scripts/deploy-unattend.sh: the only route for an answer file to a PXE
  server. Lints first and refuses on any error, backs up, swaps
  atomically, lints the live copy and restores the backup if that fails.
- .githooks/pre-commit (git config core.hooksPath .githooks): refuses a
  commit whose staged unattend fails the linter. Verified by planting a
  352-char Path (blocked) and against the exact a94c203 file (rejected).
- Boot menu: also accept SecureBoot read as hex 01, and show the raw values
  in the warning so a wrong read can be diagnosed from the screen.
2026-10-14 17:14:18 -04:00
cproudlock
454e30ff6f Floor shows a staged BIOS update from the start of WinPE; NIC speed past 4.29 Gb/s
- winpe-status-push.ps1: check-bios.cmd stages the firmware update before
  the imaging menu, but startnet's 'BIOS firmware update' push only comes
  at the end of WinPE, so builds showed a plain 'Applying image' for the
  whole apply. The first push now reports the staged update when
  X:\bios-fired.flag exists, and the floor shows 'Applying image - BIOS
  update staged' (do not power off during the next reboot). Also: CIM
  fallback for the serial where wmic is missing, and standard/engineer
  always report 5 steps.
- Network pill: cdc_ncm (e.g. the WisdPI RTL8157 5G adapter) keeps link
  speed in a 32-bit bits/s field, so 5 Gb/s reads 705 Mb/s; show the real
  speed with a note.

Tested in the win11 VM (with and without the BIOS flag, and the late
startnet push).
2026-10-14 16:56:46 -04:00
cproudlock
0d69222f85 Overview: network status pill for the two NICs
A 'Network' pill next to 'Services' lists each physical NIC with its role
(imaging / GE network, from /etc/pxe-server/nic.conf or dnsmasq's
interface= pin), link, speed, addresses, and whether dnsmasq serves
DHCP/PXE on it. It reads pxe-nic-check's verdict file
(/run/pxe-nic-status.json) when present, and shows a red banner when the
imaging NIC is on the GE network or a network with another DHCP server,
or when dnsmasq answers DHCP on a non-imaging NIC. Without the file it
shows link and addresses only. Read-only.
2026-10-14 16:49:32 -04:00
cproudlock
63fafbfc73 Boot menu: Secure Boot check for WinPE, Blancco through the shim; tracked as a file
Live since 2026-10-14 for every client (all UEFI clients now run iPXE 2.0):
- WinPE skips the 'Secure Boot must be ENABLED' warning and its 5 s wait
  when iPXE reads Secure Boot as on (${efi/SecureBoot:uint8}); otherwise,
  or on an iPXE that cannot read it, it warns as before.
- Blancco and Blancco debug start through Ubuntu's shim (iPXE shim
  command), so they run with Secure Boot on; an iPXE without the command
  boots the kernel directly as before. The separate Secure Boot TEST entry
  is gone.
- Blancco has its own menu section; only Clonezilla and Memtest stay under
  'Secure Boot must be DISABLED'.

The playbook now installs playbook/ipxe/GetPxeScript.aspx instead of an
inline copy that had drifted from the live menu. Checked that the Blancco
Releases page still parses it (one item blancco, :blancco_efi block).
2026-10-14 16:43:54 -04:00
cproudlock
045a1fb97f Secure Boot iPXE chain is the default for every UEFI client
Live since 2026-10-14: dnsmasq hands every UEFI client ipxe-shim.efi
(iPXE's shim, dual-signed under Microsoft UEFI CA 2011 + 2023), which
loads ipxe.efi from the TFTP root, now the iPXE v2.0.0 snponly build
(firmware network and USB drivers; the full build kills USB keyboards).
The per-MAC test tags and folders are gone; the old Broadcom-signed
1.21.1 build stays as ipxe-bcm.efi, the rollback.

- playbook: pinned sha256 vars, the new dnsmasq default line, and a
  verified, non-optional task that installs ipxe-shim.efi, ipxe.efi
  (snponly), ipxe.pxe and ipxe-bcm.efi (was: copy ipxe.efi and the never
  served undionly.kpxe, errors ignored).
- scripts/fetch-ipxe.sh downloads the pinned tarball and shim into
  boot-files/ for the USB build; README/SETUP list the step.
- iPXE page: ipxe.efi is staged from snponly.efi, a build that drives USB
  itself is refused, and the snponly-shim/snponly root files are gone.

Tested: the playbook task against the real release files (installs the
live hashes; a tampered file stops it before anything is replaced),
ansible --syntax-check, staging, and the full-build refusal.
2026-10-14 16:37:49 -04:00
cproudlock
a94c20331a Imaging floor: Standard/Engineer builds report progress past WinPE
The floor's 8 stages are the shopfloor pipeline, and only shopfloor-setup
reports after WinPE, so every gea-standard / gea-engineer build sat at
1/8 forever with no type.

- winpe-status-push.ps1 reads the image from the Z: share mapping when no
  PCTYPE is set and reports those builds as 5 steps.
- Send-ImagingStage.ps1 (deployed to _shared\Applications\extra\pxe-status)
  posts steps 2-5 from the unattend: start and end of specialize, first
  logon, and setup complete (succeeded). It is copied to
  C:\ProgramData\PxeStatus first, because the bulk-enrollment task deletes
  C:\Deploy before first logon. Best-effort: always exits 0.
- Floor rows use Standard/Engineer labels and show the image as the kind.

Tested as SYSTEM in the win11 VM (PowerShell 5.1): the exact unattend
command line, stages 3-5 against a test server, a dead port (exit 0), and
the image regex.
2026-10-14 16:23:52 -04:00
cproudlock
81c2d85301 Webapp: iPXE page stages the dual-signed shim, knows snponly and per-MAC folders
Staging fetches ipxe-shimx64.efi from the ipxe/shim release (sha256 from
GitHub), and uses it instead of the tarball's 2011-only shim when it is
signed under the Microsoft UEFI CA 2023 and trusts the same iPXE CA;
otherwise the tarball shim stays, with the reason shown. snponly.efi and
snponly-shim.efi are managed alongside ipxe.efi and ipxe-shim.efi, each
iPXE build says whether it drives USB itself (kills USB keyboards), and
tftp-unique-root per-MAC folders are listed read-only.
2026-10-14 15:37:14 -04:00
cproudlock
4db9dc093f Webapp: Last 3 months evidence range, keep popovers on screen
Evidence pack gets a Last 3 months quick range (three full calendar
months). Opened menus are nudged back inside the window, so header
popovers no longer start off-screen on phones or a wrapped header.
Buttons and filter pills inside a menu keep their own styling.
2026-10-14 15:12:38 -04:00
cproudlock
2dbaaded44 Webapp: evidence pack date range, US month labels, hardware gap rows
Evidence pack takes ?from=&to= (inclusive, Eastern time) with From/To
pickers and quick ranges; ZIP is named ENT-056_<site>_<from>_to_<to>.
Month keys show as 'September 2026'. Hardware gap rows on the to-do
use the shared one-line grid with chips for the missing pack or BIOS.
2026-10-14 15:08:36 -04:00
cproudlock
a54df7ebd9 Webapp redesign: imaging floor at laptop widths, keyboard tips, placeholders
From the design review:
- Floor rows reserve 150px for actions instead of 190px; stage text is
  one line with the full text in a tip (safety text still wraps); the PC
  type hides first; rows stack only below 960px of panel width, with the
  actions on the status line instead of their own row.
- Tips also show on keyboard focus.
- Placeholders use the muted token (dark-theme contrast was 3.6:1).
- Save bar focus ring uses the save bar's accent (2.8:1 before).
2026-10-14 14:52:45 -04:00
cproudlock
2ed2cbe2e9 Webapp redesign: fixes from the functional regression test
- Unattend: a command's '...' menu stayed open after Move up/down and
  covered the next row's menu, so the next click could stage removing the
  wrong command. The menu now closes and focus returns to its button.
- '...' menus close on an outside click, on Escape and when another
  opens (static/app.js), for every page.
- Drivers: Remove asks to confirm again, as before the redesign. Long zip
  paths in a row body wrap (the page scrolled sideways at 390 px).
- Rollback review returns to the page it was opened from (Dell page or
  Updates overview), for both Cancel and the rollback itself.
- To-do: the ServiceNow period offers a year again (366 days).
- Imaging floor: a search that matches a log-only bay opens that group;
  its open state survives the live refresh. 'Clear all' states the
  number of session files it deletes, hidden finished ones included.
- Dashboard: the image key pattern is valid under the browser's v flag
  (client-side validation was silently skipped).
- Import: allowed_import_source() compares real paths, so
  '<upload dir>/../..' is refused. This path traversal predates the
  redesign; the import moves the source tree, so it mattered.
2026-10-14 14:48:10 -04:00
cproudlock
2bcc9d8016 Webapp redesign: fixes from the code review
Blocker
- /reports/export returned 500: REPORT_EXPORT_RANGES and _months_ago were
  deleted with the old reports route. Restored.

Major
- A session without stage_index/stage_total took down /, /imaging and
  /imaging/tiles (int filter on Undefined). The templates default both;
  saving a LAPS password for a session that no longer exists is refused
  (it created a bare session).
- /reports/todo parsed every report four times (23 s at 700 reports).
  blancco_report.parse() now caches by path, mtime and size: 0.09 s.
- Stored XSS through the session serial in the Clear confirm (the
  browser decodes &#39; before running an inline handler). The message
  moves to data-confirm, and update_session() only accepts serials of
  letters, digits, '.', '_' and '-' (400 from /imaging/status otherwise).
- Reports without a document id and with odd characters in the file name
  got a key the mark routes rejected, so they could never be cleared.
  report_key() now always produces a valid key.

Minor
- failure_note() handles a list-valued erasure target (500 on the report
  and to-do pages).
- _recently_marked() tolerates naive, 'Z' and non-dict state entries.
- Key regexes use fullmatch (a trailing newline was accepted).
- Unreadable reports are no longer counted as to tick / to destroy; the
  reports page already flags them as Unreadable.
- Rollback review recounts restorable files after reading the backups.
- Enrollment: the startnet fallback package (no or empty ppkg.conf) and a
  quoted SOURCE_PPKG are protected too.
- Backup upload overwrite check is case-insensitive (SMB share) and
  replaces the existing file rather than adding a case twin.
- Overview polling keeps the last good rows on an error and pauses while
  the tab is hidden. A shown or typed LAPS password survives the floor's
  refresh.
- hardware_dismiss only redirects to same-site paths; tiles ?limit is
  clamped; the Updates overview survives an unreadable Dell scan or iPXE
  state; describe() uses aversion(); a client cannot label its session
  'inferred'.
2026-10-14 14:42:21 -04:00
cproudlock
fd00d5ff89 Webapp redesign phase 6: the remaining pages
Restyled to the same row pattern (one line, details in an expandable
body, one action plus a menu, tokens for light and dark):
- hardware coverage, audit log (family chips, destructive actions in
  red, client-side filters, first 300 rows), Clonezilla backups,
  enrollment packages (active package marked and undeletable, MCL flags
  with a visible legend)
- image Overview, OS and packages, Unattend (sticky save bar), Import
- Dell, iPXE, Blancco and BPRT update pages; Dell history links to the
  rollback review instead of rolling back on one click
- boot menu editor (tabs, rows with a menu, lint chips that jump to the
  line, tokens instead of hard-coded dark colours) and the imaging
  session detail page

Fixes found on the way:
- BPRT batch Activate never asked to confirm: a hidden input named
  'confirm' shadowed window.confirm in the inline handler, so the form
  submitted without a prompt. It now calls window.confirm.
- Backup upload refused when a backup with that name exists, unless
  'Replace the existing backup' is ticked (it overwrote silently).
- app.js: em dashes in comments replaced (plain ASCII).
2026-10-14 14:29:03 -04:00
cproudlock
a6f95fcca1 Webapp redesign phase 5: Blancco reports, report and to-do
Reports (/reports):
- Grouped by month of the erasure date inside each report (not the file
  date), newest first; older months fold. Each month header shows the
  evidence pack state (builds on the 1st, to upload with Download, or
  uploaded).
- One line per report: erasure date, system serial, model, drives,
  result ('Successful, fallback' when a failed attempt fell back to
  another standard), cloud and ServiceNow state. The row body lists every
  drive and erasure attempt with the failure reason.
- Search by system serial, model, drive serial or file; filters for
  failed, not in cloud and to tick. 'To tick' uses the to-do's 30-day
  window; older unticked reports read 'Not ticked'.
- Export and the evidence pack download moved into header popovers.

Report (/reports/view/<file>):
- Header with serial, model and the per-drive result; a notice names
  the drive to destroy until its COD is recorded; Copy ServiceNow note.
- One card per drive with every attempt, a step track and the failure
  reason; steps and error counts fold. System, customer and provenance
  (integrity signature folded, copyable) on the side. Sizes in decimal
  units, as on the list.
- blancco_evidence.failure_note() is shared with the ServiceNow list.

To-do (/reports/todo):
- The ServiceNow tick list is now its first section (period and show
  ticked filters); /reports/servicenow redirects there.
- Five counters; one line per item with details in the row body.
- COD recorded, Mark uploaded and Resolved confirm first (the attribute
  quoting is safe for JSON) and can be undone for 24 hours from the
  page. The header total includes hardware gaps, matching the sidebar.
2026-10-14 14:00:54 -04:00
cproudlock
cc1e76f3ab Webapp redesign phase 4: Updates overview and rollback review
- GET /updates/: one card per updater (Dell, iPXE, Blancco, BPRT) with
  its state, a Check now / Open action and details in accordions; the
  shared download cap is a small popover.
- Dell applies list: one line per apply titled by its first change
  ('Latitude 5350 pack A11 to A12', model looked up in the maps) and
  'and n more'; the chip says how many written files are unchanged since
  the apply. The body lists every change, skipped item, error and file.
- dell_updates.file_status(): per file an apply wrote, unchanged /
  changed / missing by content hash, the same test rollback uses.
  describe(), history_view() and rollback_preview() build on it.
- GET /updates/dell/rollback/<id>: review a rollback before doing it,
  with the files that would be restored and a diff of each. The POST
  route is unchanged; both forms can return to the overview.
2026-10-14 13:54:41 -04:00
cproudlock
731d7b044d Webapp redesign phase 3: Drivers tab
- One line per entry: position, model tokens, pack, OS, size and status
  chips; family, friendly name, osId, folder, zip state, Dell hint and
  the verbatim lint findings are in the row body.
- Status chips compare the staged map with the saved one: New, Edited,
  Moved from #n, Removing, the worst current finding, and 'Clears on
  save' / 'Clears n' when the staged changes resolve this entry's
  findings. The toolbar and a sticky save bar show lint now and after.
- Removing an entry is staged with Undo remove instead of deleting the
  row; Undo edit puts an edited entry back.
- Reordering moved into the row menu (Move up / down / to position).
  The Move dialog shows the order after the move and re-lints the staged
  map with the move applied, listing any new errors before staging it.
- An edit that changes the tokens but keeps the old friendly name is
  flagged in the row and in the review step.
- Filters: Win11/Win10 with counts, 'Findings or unsaved', 'Zip
  problems' (missing, trap and case); a status key explains the chips.
- _image_header.html: new header and underline tabs for every image tab;
  the shared review modal and diff use the design tokens.
2026-10-14 13:51:04 -04:00
cproudlock
0b86255711 Webapp redesign phase 2: Overview and imaging floor
Imaging floor (/imaging, /imaging/tiles):
- One line per PC: status chip, serial with machine number, hostname
  and IP, stage text with a segmented stage track, one action plus a
  '...' menu. Everything else (description, error, timestamps, Intune
  ID + QR, Set category / ARTS links, LAPS QR, log tail) is in the row's
  expandable body.
- Order: failed, needs action (stage 7), in progress, done; log-only
  bays fold into one collapsed group. imaging_status.floor_state() and
  floor_order() compute this.
- Status filter buttons with live counts next to the text filter. SSE
  refresh, expanded-row memory and LAPS handling are unchanged.
- Stage text is never truncated; BIOS updates and erasing are bold.

Overview (/):
- Services popover, technician to-do line, the six most urgent floor
  rows (?limit=6 partial, refreshed every 15 s), and image cards with
  lint warnings, map errors and missing zips as separate chips. New,
  clone, delete and unattend stay on the cards' menus.

Copy buttons are now handled once in static/app.js, with a fallback for
plain-http pages where navigator.clipboard is unavailable.
2026-10-14 13:44:07 -04:00
cproudlock
35cad62733 Webapp redesign phase 1: shell, tokens and shared components
- static/pxe-ui.css: the redesign's colour tokens (light and dark),
  shell, chips, stage track, expandable rows, menus and buttons. The
  legacy tokens (--bg-card, --border, --text-light, ...) alias onto the
  new ones so pages not yet redesigned blend in.
- base.html: new sidebar (Operate, Images, Updates, Tools) with the GE
  Aerospace logo on the brand blue, to-do count, theme toggle with
  aria-pressed, and a menu button on narrow screens. The to-do banner is
  now a one-line strip. Skip link and <main> landmark added.
- _ui.html: Jinja macros for chip, track, the actions menu and the page
  header.
2026-10-14 13:38:05 -04:00
cproudlock
dfd57bef17 Webapp: fix six live bugs found in the page inventory
- Report view: the hero summarised only the first erasure, so a drive whose
  Purge failed and Clear fallback succeeded showed Failed while the list said
  Successful. It now uses the per-drive outcome (report_result) and lists the
  standards that wiped each drive.
- To-do and ServiceNow: reports with no usable document id could never be
  ticked or cleared (400). They are now tracked by file name (file:<name>).
- Imaging: the LAPS password field is masked, with a Show/Hide toggle.
- Imaging: succeeded sessions not updated for ACTIVE_WINDOW_HOURS are hidden
  from the dashboard (failed and in-progress always stay), with Show all.
- Enrollment: the active SFLD package can no longer be deleted, which left
  ppkg.conf pointing at a missing file; its row is marked Active.
- Import: confirm before importing, saying upload-folder sources are moved
  and same-name files in the image are overwritten.
2026-10-14 13:29:00 -04:00
cproudlock
24b4d9df66 Dell updates: say when the results predate the last apply
After an apply the tables still showed the check from before it, so a pack
just applied looked like it still needed updating until the next check.
2026-10-13 17:38:14 -04:00