Commit Graph

389 Commits

Author SHA1 Message Date
cproudlock
ead3314f58 Two-NIC standard: pin imaging and GE NICs by MAC everywhere
Every server has an imaging NIC (static, the only one that serves
DHCP/TFTP/HTTP boot/SMB) and a GE corporate NIC (DHCP client, SSH and the
webapp on 9009). The playbook guessed the PXE NIC and the installer put
172.16.9.1/24 on every en* NIC, so a two-NIC box could answer on the GE
network.

- New per-site vars pxe_nic_mac / corp_nic_mac (required, from
  playbook/site-vars.yml or -e; template site-vars.example.yml), stable
  names pxe_nic_name / corp_nic_name (default pxe0 / corp0, overridable),
  pxe_server_ip, pxe_subnet, DHCP range. pxe_imaging_mac / pxe_ge_mac are
  derived from them. The play stops before touching anything if a MAC is
  unset, malformed, duplicated or not on the box (lists the box's NICs).
- Heuristic NIC detection and the single-NIC netplan task replaced by a
  two-NIC netplan template (match macaddress + set-name), installed by
  pxe-netplan-apply.sh: validate in a scratch root, back up, generate,
  rename, apply, restore the previous config on any failure. cloud-init
  network config disabled so it cannot rewrite it.
- dnsmasq: /etc/dnsmasq.d/pxe-nic-only.conf from the vars (interface=,
  except-interface=, no-dhcp-interface=); dnsmasq.conf IP/range from vars.
- UFW per interface: imaging NIC 22, 67/udp, 69/udp, 80, 4433, 445, 139,
  137/udp, 138/udp, 9009; GE NIC 22 and 9009 only. Old any-interface rules
  and the stale 9011 rule removed after the new ones are in.
- sysctl drop-in: IPv4/IPv6 forwarding off; ip_nonlocal_bind so Apache
  can bind the imaging IP before the address is up.
- Samba [global]: interfaces = 127.0.0.0/8 <imaging NIC>, bind interfaces
  only. Apache 80 and 4433 on the imaging IP only; 9009 stays on all (TODO:
  the webapp has no auth).
- autoinstall: DHCP only on en*, no static 172.16.9.1; apt geoip off and
  offline-install fallback now that the corp NIC may have a lease.
- Blancco UEFI/Secure Boot: blancco_kver (7.0.0-34-generic) drives the
  kernel, initramfs and modules builds instead of the running kernel;
  outputs vmlinuz-ubuntu-70, kexec-initrd-70.img, kmod-<kver>.tar.gz as the
  menu and blancco-init.sh expect; creates: guards keyed on the full kver.
  Ubuntu's shimx64.efi.signed published as blancco/shimx64.efi.
  download-packages.sh adds shim-signed, linux-generic-hwe-24.04 and the
  pinned linux-image/modules/modules-extra for BLANCCO_KVER.

Rendered with the live MACs and names, netplan generate output is
identical to the live config; pxe-nic-only.conf and the Samba binding
match live directive for directive.
2026-10-15 08:53:41 -04:00
cproudlock
689eebf9d1 Merge branch 'nic-guard' into pxe-webapp-redesign 2026-10-15 08:40:57 -04:00
cproudlock
d3a0f90ca1 Re-imaging the same PC: clear its stale SMB/conntrack state every time
A PC whose imaging failed and is re-imaged gets the same IP, and the
server still held state from the dead run, so net use failed (System
error 53). The April fix (18537ac) did not cover it:

- The dnsmasq hook only acted on 'add'/'del'. A re-image comes back to the
  existing lease, which dnsmasq reports as 'old', and only when the
  hostname/MAC changed. Now dnsmasq sets script-on-renewal, and the hook
  clears state on 'old' when the request comes from PXE firmware or iPXE
  (vendor class PXEClient), i.e. the machine just network-booted. A
  Windows lease renewal (MSFT 5.0) is left alone.
- The 1 h conntrack timeout never applied after a reboot: systemd-sysctl
  runs before ufw loads nf_conntrack, so the keys did not exist and the
  5-day default stayed. /etc/modules-load.d/pxe-conntrack.conf loads it
  first.
- smb-soft-reset.sh flushed neighbours on br-pxe, which no longer exists.

Live 2026-10-14. Hook tested on the server with an unused IP: network-boot
renewal clears, Windows renewal and no vendor class do not, add clears.
2026-10-14 17:19:33 -04:00
cproudlock
bc422649e9 NIC guard: dnsmasq never serves DHCP/PXE on the GE network
pxe-nic-check classifies each NIC from what it hears: LLDP/CDP/802.1X
from managed switches, foreign DHCP offers and their domain (one
DHCPDISCOVER on the imaging NIC; the GE NIC's networkd lease is read
instead), STP and ARP hosts for information. A kernel BPF filter keeps
bulk SMB traffic out of the capture loop; bridge ports are listened on
too, since LLDP and EAPOL stop at the port.

- dnsmasq drop-in runs --guard as ExecStartPre: exit 1 on a ge or
  foreign-dhcp imaging verdict, fail open (loudly) on bugs or no link
- pxe-nic-check.timer runs --watch every 5 min: stops dnsmasq on a bad
  imaging verdict, never starts it
- /run/pxe-nic-status.json (0644, atomic) for the webapp
- playbook installs it all and writes /etc/pxe-server/nic.conf from
  pxe_imaging_mac, pxe_ge_mac, pxe_corp_domain_suffixes,
  pxe_nic_allowed_switches
- unit tests plus a no-root network-namespace lab (veth pairs, a raw
  socket DHCP/LLDP peer and real dnsmasq); docs/pxe-nic-guard.md has the
  operator notes and the two floor tests
2026-10-14 17:18:53 -04:00
cproudlock
28fd9fe474 Fix the unattend that broke Engineer builds; gate unattend deploys on the linter
a94c203 added a RunSynchronous Path of 417 chars to both std/eng answer
files. Windows caps it at 259 and then rejects the WHOLE answer file, so
an Engineer build stopped at 'Windows could not parse or process unattend
answer file'. The live files were restored to their pre-a94c203 versions
the same day.

- The status steps now fit: Send-ImagingStage.ps1 -Persist copies itself
  out of C:\Deploy (no cmd copy chain), short PowerShell switches, default
  arguments left out. Longest Path is 168 chars; lint-unattend passes.
  NOT deployed: these files must go live only through deploy-unattend.sh.
- scripts/deploy-unattend.sh: the only route for an answer file to a PXE
  server. Lints first and refuses on any error, backs up, swaps
  atomically, lints the live copy and restores the backup if that fails.
- .githooks/pre-commit (git config core.hooksPath .githooks): refuses a
  commit whose staged unattend fails the linter. Verified by planting a
  352-char Path (blocked) and against the exact a94c203 file (rejected).
- Boot menu: also accept SecureBoot read as hex 01, and show the raw values
  in the warning so a wrong read can be diagnosed from the screen.
2026-10-14 17:14:18 -04:00
cproudlock
454e30ff6f Floor shows a staged BIOS update from the start of WinPE; NIC speed past 4.29 Gb/s
- winpe-status-push.ps1: check-bios.cmd stages the firmware update before
  the imaging menu, but startnet's 'BIOS firmware update' push only comes
  at the end of WinPE, so builds showed a plain 'Applying image' for the
  whole apply. The first push now reports the staged update when
  X:\bios-fired.flag exists, and the floor shows 'Applying image - BIOS
  update staged' (do not power off during the next reboot). Also: CIM
  fallback for the serial where wmic is missing, and standard/engineer
  always report 5 steps.
- Network pill: cdc_ncm (e.g. the WisdPI RTL8157 5G adapter) keeps link
  speed in a 32-bit bits/s field, so 5 Gb/s reads 705 Mb/s; show the real
  speed with a note.

Tested in the win11 VM (with and without the BIOS flag, and the late
startnet push).
2026-10-14 16:56:46 -04:00
cproudlock
63fafbfc73 Boot menu: Secure Boot check for WinPE, Blancco through the shim; tracked as a file
Live since 2026-10-14 for every client (all UEFI clients now run iPXE 2.0):
- WinPE skips the 'Secure Boot must be ENABLED' warning and its 5 s wait
  when iPXE reads Secure Boot as on (${efi/SecureBoot:uint8}); otherwise,
  or on an iPXE that cannot read it, it warns as before.
- Blancco and Blancco debug start through Ubuntu's shim (iPXE shim
  command), so they run with Secure Boot on; an iPXE without the command
  boots the kernel directly as before. The separate Secure Boot TEST entry
  is gone.
- Blancco has its own menu section; only Clonezilla and Memtest stay under
  'Secure Boot must be DISABLED'.

The playbook now installs playbook/ipxe/GetPxeScript.aspx instead of an
inline copy that had drifted from the live menu. Checked that the Blancco
Releases page still parses it (one item blancco, :blancco_efi block).
2026-10-14 16:43:54 -04:00
cproudlock
045a1fb97f Secure Boot iPXE chain is the default for every UEFI client
Live since 2026-10-14: dnsmasq hands every UEFI client ipxe-shim.efi
(iPXE's shim, dual-signed under Microsoft UEFI CA 2011 + 2023), which
loads ipxe.efi from the TFTP root, now the iPXE v2.0.0 snponly build
(firmware network and USB drivers; the full build kills USB keyboards).
The per-MAC test tags and folders are gone; the old Broadcom-signed
1.21.1 build stays as ipxe-bcm.efi, the rollback.

- playbook: pinned sha256 vars, the new dnsmasq default line, and a
  verified, non-optional task that installs ipxe-shim.efi, ipxe.efi
  (snponly), ipxe.pxe and ipxe-bcm.efi (was: copy ipxe.efi and the never
  served undionly.kpxe, errors ignored).
- scripts/fetch-ipxe.sh downloads the pinned tarball and shim into
  boot-files/ for the USB build; README/SETUP list the step.
- iPXE page: ipxe.efi is staged from snponly.efi, a build that drives USB
  itself is refused, and the snponly-shim/snponly root files are gone.

Tested: the playbook task against the real release files (installs the
live hashes; a tampered file stops it before anything is replaced),
ansible --syntax-check, staging, and the full-build refusal.
2026-10-14 16:37:49 -04:00
cproudlock
a94c20331a Imaging floor: Standard/Engineer builds report progress past WinPE
The floor's 8 stages are the shopfloor pipeline, and only shopfloor-setup
reports after WinPE, so every gea-standard / gea-engineer build sat at
1/8 forever with no type.

- winpe-status-push.ps1 reads the image from the Z: share mapping when no
  PCTYPE is set and reports those builds as 5 steps.
- Send-ImagingStage.ps1 (deployed to _shared\Applications\extra\pxe-status)
  posts steps 2-5 from the unattend: start and end of specialize, first
  logon, and setup complete (succeeded). It is copied to
  C:\ProgramData\PxeStatus first, because the bulk-enrollment task deletes
  C:\Deploy before first logon. Best-effort: always exits 0.
- Floor rows use Standard/Engineer labels and show the image as the kind.

Tested as SYSTEM in the win11 VM (PowerShell 5.1): the exact unattend
command line, stages 3-5 against a test server, a dead port (exit 0), and
the image regex.
2026-10-14 16:23:52 -04:00
cproudlock
aa026bace0 BIOS manifest: replace the 24250 line with QB24250, QC24250 and QC24251
'24250' also matched the Dell 24 All-in-One EC24250, which needs a different
BIOS (Dell_EC27250_EC24250). The three exact models are the ones the combined
file is built for. Found by the webapp's Dell check (ambiguous match).
2026-10-13 17:23:57 -04:00
cproudlock
ae806cf98e CMM imaging: goCMM 2.17.0.0 instead of 2.12.3
FileVersion detection on goCMM.exe so an older 2.x does not satisfy it. Same
NSIS installer and /S switch. Deployed to installers-post/cmm and the share's
shopfloor-setup copy; 2.12.3 moved to installers-post/cmm-archive so it is not
copied to every CMM image.
2026-10-13 17:10:09 -04:00
cproudlock
28cd977c9e goCMM restore: drop the 1.1 Installation Directory value on 2.x PCs
Per-bay backups were taken under goCMM 1.1, so goCMM.reg restores
Installation Directory = the 1.1 folder. goCMM 2.x never reads it, but the 2.x
installer sees it as '1.1 installed' and shows a dialog /S cannot answer,
which hung GE-Enforce on CMM11 and CMM12. Kept when 1.1 really is installed.
2026-10-13 16:58:55 -04:00
cproudlock
bfd5c5b75e Blancco init: optional blancco.base= so a staged release can be test-booted
Default is unchanged. Takes effect only after kexec-initrd-70.img is rebuilt.
2026-10-13 16:58:55 -04:00
cproudlock
dd9ee33cd5 Playbook: tools and Zscaler root CA the webapp Updates pages need
sbsigntool, zip and curl, the Zscaler root CA in the system store (GitHub is
TLS-intercepted on the corporate NIC; downloads.dell.com is not), and
/var/lib/pxe-webapp/updates.
2026-10-13 16:58:55 -04:00
cproudlock
ddf0a2fdf2 BIOS manifest: update to Dell's current releases, fix wrong-platform lines
Scanned against Dell's per-model catalogs (CatalogIndexPC, 2026-10-09).
About 40 lines were behind, nine pointed at another platform's BIOS
(Latitude 7450 -> OptiPlex 7450, Latitude 5530/5540 -> Precision
5530/5540, Precision 5490/7770/7780 -> OptiPlex AIO, Precision 5550 ->
Latitude 5550, OptiPlex 7400/7410/7420 AIO -> Latitude 7x00/7x10/7x20),
and '7320 2-in-1 Detachable' never matched the WMI name. Specific lines
now precede general ones (5430 Rugged, 7090 Ultra, 3000 Thin Client)
because check-bios.cmd takes the first substring match.
2026-10-12 17:02:47 -04:00
cproudlock
6fac73c947 startnet: map Y: before status push and preflight on every path
The gea-standard/gea-engineer path (purpose -> tenant -> enroll_staged)
never mapped the enrollment share, so winpe-status-push.ps1 and
preflight.ps1 silently skipped those builds. Map it at :end if absent.
2026-10-09 09:59:11 -04:00
cproudlock
f0c049376c Report hardware from the BIOS check, so every WinPE path is covered
The hardware report went out from winpe-status-push.ps1 on the
enrollment share (Y:), but startnet.cmd never maps Y: on the
gea-standard / gea-engineer path (choices 1/2 go purpose -> tenant ->
enroll_staged), so those builds never reported. The same gap means
neither the status push nor preflight.ps1's driver check has ever run
for standard/engineer images.

The BIOS share is mapped by every WinPE boot before the menu, and
check-bios.cmd lives on it, so it now calls hardware-report.ps1 from
the same folder (best effort, output discarded). winpe-status-push.ps1
goes back to status only, so shopfloor bays are not counted twice. The
playbook deploys hardware-report.ps1 with the other BIOS files.
2026-10-09 09:46:11 -04:00
cproudlock
b691a82530 Hardware coverage: flag models the server has no drivers or BIOS for
WinPE now reports each machine's WMI model, manufacturer, Dell system
SKU and BIOS version to /hardware/seen (winpe-status-push.ps1, best
effort, wmic only). The webapp records every model seen and checks it
against what the server has:
  - each image's Deploy/Control/HardwareDriver.json, matched exactly as
    PESetup's GetDriverByModel does (as reimplemented in
    lint-driver-catalogue.py), plus whether the pack's zip is on disk;
  - the WinPE BIOS manifest (substring match like check-bios.cmd).

/hardware lists every model with per-image driver status and BIOS
status. A model missing a Win11 pack or a BIOS entry becomes item 5 on
the Technician To-Do (and the banner count) until the pack/BIOS is added
or it is dismissed. Virtual machines are ignored.

Also deploys the repo winpe-status-push.ps1 over the live copy, which
predated the -CurrentStage/-StageIndex parameters startnet.cmd passes
for the BIOS-flash stage, so that status post was failing.
2026-10-09 09:23:52 -04:00
cproudlock
83ba4865e8 Show every time in US Eastern (America/New_York)
Webapp: config.DISPLAY_TZ (default America/New_York) and a local_dt
filter that renders datetimes, ISO strings and Unix timestamps as e.g.
"2026-10-05 13:54 EDT", DST included. Used for the ServiceNow tick list,
technician to-do, imaging tiles and detail page; timestamp_fmt (reports,
backups, enrollment) goes through it too instead of relying on the
process TZ. CSV report_date is Eastern with its offset. Evidence-pack
months are Eastern calendar months, so a wipe late on the 31st lands in
that month rather than the next.

Blancco: init points the new root's /etc/localtime at America/New_York
so the GUI clock is local. Report timestamps keep their explicit
offset; report.utc_offset stays 0 since a fixed offset cannot follow DST.
Report file names stay UTC, as Blancco's own Save names them.
2026-10-05 14:16:14 -04:00
cproudlock
8c36d1ba5d blancco-init: post_status uses sed, not tr
The initrd's busybox has the tr applet but no /bin/tr link, so the
quiet boot printed "/init: line 140: tr: not found" and the stage 1/2
dashboard posts were skipped. sed is linked.
2026-10-05 14:01:26 -04:00
cproudlock
12f6f00cdb blancco-progress: take the final state from the report file
First normal (non-debug) run stuck at 6/7 "Report sent to cloud": outside
Blancco's debug mode Albus does not log erasureFinishedForEntity, Report
saving success or "Auto process reached the final step", which the
parser had been taken from a debug-mode journal.

A watcher thread now reads each new /reports/*.xml (auto mode always
saves there) once it is stable and posts 7: "Wipe complete" (succeeded)
or "Erasure FAILED: <model serial, ...>" (failed). Judged per drive, so
a failed Purge with a successful Clear fallback on the same drive is a
wipe. --outcome FILE prints the verdict for a report.
2026-10-05 13:59:09 -04:00
cproudlock
44a7eeae95 blancco: CMMC evidence pack, ServiceNow tick list, cloud-upload status
Automates the Blancco parts of the onsite-support CMMC playbook:

- ENT-056/158 evidence pack: /reports/evidence?month=YYYY-MM returns
  ENT-056_<Site>_<YYYY-MM>.zip (summary.csv, README, each report's XML
  and PDF). pxe-blancco-evidence.timer saves last month's pack to the
  reports share's evidence/ folder on the 1st (Persistent).
- ENT-045/159 ServiceNow tick list: /reports/servicenow lists recent
  reports as "Wiped" or "Blancco Wipe failed" with a pre-written
  destruction note, copy buttons and Done (CSRF-protected).
- Cloud upload: blancco-progress.py posts each report's upload result
  ("Sending report document id" -> "Report sent to MC") to
  /reports/cloud-status (CSRF-exempt like /imaging/status); the Reports
  page and the CSVs show it, so a wipe missing from BMP is visible.
- CSV gains failure_reason (exception messages, failed steps, error
  counts), business_location and cloud_upload.
- Preferences: business_location "West Jefferson" (config.BLANCCO_SITE)
  and pdf+xml reports; the report copier now copies the PDF too, named
  after its XML. .part and hidden files are not listed as reports.

State (cloud confirmations, ServiceNow ticks) is a JSON file at
/var/lib/pxe-webapp/blancco-state.json, off the report share.
2026-10-05 13:43:16 -04:00
cproudlock
19049e6b1f blancco: live erase progress on the /imaging dashboard
Each Blancco PC now gets a tile on /imaging, keyed by BIOS serial with
pctype "blancco", using the /imaging/status endpoint the WinPE clients
already post to (stage_total 7):

  1 Downloading Blancco, 2 Starting Blancco   (blancco-init.sh, wget)
  3 Waiting for licence / Licence OK (N left)
  4 Erasing: step N - <step> [NN%]
  5 Erasure finished / Report saved
  6 Report sent to cloud
  7 Done (succeeded), or Erasure FAILED (failed, sticks)

playbook/blancco-progress.py follows the AlbusSeven journal inside the
Blancco rootfs and maps its lines to those stages; patterns taken from
a real 7.20 auto-mode run and replayed with --replay/--dry-run, plus a
synthetic failed step. Init fetches it from the web root at boot and
installs it as blancco-progress.service; the playbook deploys it there.
The report copier adds a "Report copied to share" log line. Stage 1
resets a finished tile for the same serial. All posts are best effort.
2026-10-05 12:50:40 -04:00
cproudlock
a97faf9fad blancco-init: auto-save reports to the SMB share in process=auto
process=auto erases unattended but Albus forces auto_backup off outside
workflow/manual, so every report needed a manual Save to reach the
share. Debug SSH on a live 7.20 run showed Albus does keep each report
at /reports/<document_id>.xml ("Report saving success", "CRC verified")
and uploads it to MC, with /mnt/share already mounted.

Install blancco-report-sync as a systemd service in the new root: every
10s copy each finished /reports/*.xml (older than 5s, not yet copied)
to /mnt/share as YYYYMMDD_HHMMSS_report.xml (UTC, as a manual Save
names it), via a .part file and rename. Verified live: the auto-run
report landed as 20261005_164520_report.xml.
2026-10-05 12:46:48 -04:00
cproudlock
165d6d34a3 blancco-init: debug boots open SSH for root
With blancco.debug=1 the init now also creates /etc/debug.environment in
the new root (Blancco's isDebugMode, so 2_sshd starts sshd) and installs
root's authorized_keys from http://<server>/blancco/debug_authorized_keys.
Root is locked and sshd is key-only, so normal boots stay closed and a
debug boot without that file on the server stays closed too.

Used to find where Albus keeps the finished report in process=auto, which
never auto-backs-up to the share. Live GetPxeScript.aspx has a matching
"Blancco Drive Eraser (debug / SSH)" item (UEFI, 7.0 kernel, verbose);
not in the playbook menu.
2026-10-05 12:42:10 -04:00
cproudlock
b50ff36f71 blancco-preferences: back to process=auto until MC has a default workflow
process=workflow failed on the first real run: "failed to fetch the
default workflow ... there is no default workflow for user
api.wipecheck@ge.com". The GE Blancco console needs a default workflow
assigned to that account. Until then run auto, as live 7.15 did:
unattended erasure once licensed, report to the cloud automatically,
SMB share copy via Save (auto_backup is forced off outside
workflow/manual).
2026-10-05 12:32:54 -04:00
cproudlock
143c27c3bf blancco-preferences: run the cloud workflow process
Live 7.15 ran process=auto: it erased unattended but Albus forces
auto_backup off outside workflow/manual ("... not workflow or manual:
Report auto backup must be disabled"), so every report needed a manual
Save to reach the share. 7.20 was set to manual, which honours
auto_backup but waits at "Select devices" (seen on the first real run).

workflow is the only process that both runs unattended (the account's
default MC workflow, workflow default=true, fetched over WiFi) and
auto-backs-up the report ("Start file watcher for auto backup of
workflow process"). Live preferences.xml and config.img updated
(backups .bak-20261005-pre-workflow).
2026-10-05 12:29:38 -04:00
cproudlock
b3ed4cda81 blancco-init: give Blancco /run/initramfs from config.img on UEFI
On a native boot config.img is an initrd and archiso leaves its files in
/run/initramfs, where /opt/scripts/inc/1_reportKey, 2_certificate,
1_config and 2_flexible_container install report.key, ca.pem,
hybrid.ids and the flexible container. The Ubuntu-kernel UEFI path never
created /run/initramfs, so 1_reportKey wrote an empty /albus/report.key
and the rest were skipped (true since 7.15). Fetch config.img, unpack it
on a tmpfs mounted at the new root's /run (systemd keeps an existing
/run), and fail loudly if report.key is missing.

First real 7.20 run (Dell Pro 5 14, Panther Lake, 7.0 kernel) licensed
and erased, but did not start the cloud workflow on its own or save the
report without a click; the missing report key is the leading suspect.
2026-10-05 12:24:29 -04:00
cproudlock
14eab13a1a blancco-init: per-kernel kmod tarball, lazy umount of the report share
- Fetch kmod-$KVER.tar.gz first, falling back to kmod.tar.gz, so more
  than one boot kernel can share the web root. Live now boots the UEFI
  path on Ubuntu's 24.04 HWE 7.0.0-34-generic kernel: Panther Lake
  laptops (Dell Pro 14, AX211 on CNVi) need iwlwifi firmware API 100,
  which the 6.8 driver cannot load. 6.8 stays deployed as the rollback.
- The 10s DefaultTimeoutStopSec did not reach the /mnt/share mount: a
  mount unit's timeout is TimeoutSec=, defaulting to the start timeout.
  Add a mnt-share.mount drop-in with TimeoutSec=10s, LazyUnmount=yes and
  ForceUnmount=yes.

NOT in the playbook yet: it builds the initrd and kmod tarball from the
server's running kernel (6.8). The 7.0 artifacts on 172.16.9.1 were
built off-box; installing linux-generic-hwe-24.04 on the server is what
makes a playbook run reproduce them.
2026-10-05 12:17:11 -04:00
cproudlock
783f2fece0 blancco: quiet PXE boot, full trace with blancco.debug=1
The UEFI loader ran under set -x and printed modprobe -v, ip link and
ip addr, and neither kernel cmdline had quiet, so the screen scrolled
hundreds of lines before the Blancco GUI. Now:

- quiet loglevel=3 on both Blancco kernel lines (also quiets systemd)
- init clears the screen and prints one line per step
- blancco.debug=1 on the cmdline restores the old trace
- failure paths still print their diagnostics and drop to a shell

Live GetPxeScript.aspx changed to match (backup
.bak-20261005-pre-quiet) and kexec-initrd.img rebuilt.
2026-10-05 12:05:30 -04:00
cproudlock
1f5f9eb60a blancco-init: cap systemd stop jobs at 10s
Restarting Blancco after an erasure stalled on "A stop job is running
for /mnt/share": the network goes down before the SMB report share is
unmounted, and the CIFS umount waits out the 90s default. Reports are
written long before shutdown, so set DefaultTimeoutStopSec=10s in the
switched-to rootfs.
2026-10-05 12:02:43 -04:00
cproudlock
77655bbfcd pxe: ship Ubuntu iwlwifi firmware with the Blancco kmod tarball
The UEFI Blancco path boots the Ubuntu 6.8 kernel on Blancco's rootfs, so
iwlwifi loads firmware from the Blancco image. 7.20 dropped the API 86
ucode for AX201/AX210/AX211 and ships only API 89, which the 6.8 driver
cannot use: WiFi never came up, so no cloud licence (seen 2026-10-05).

kmod.tar.gz now also carries the host's lib/firmware/iwlwifi-* (183
files, .zst, which the kernel prefers over the image's .xz). Files only:
on the Blancco rootfs lib is a symlink to usr/lib. Live tarball rebuilt
(backup kmod.tar.gz.bak-20261005-pre-iwlfw); the task's creates: guard
means an existing server needs the file removed to pick this up.
2026-10-05 12:00:24 -04:00
cproudlock
372c3802e5 pxe: Blancco initrd pulls in module deps and NIC firmware
The kexec-initrd build copied a fixed set of module directories plus
drivers/net/mii.ko. Ubuntu ships that as mii.ko.zst, so the -e test
skipped it, and usbnet, r8152 and cdc_ether then failed to load with
"Unknown symbol mii_*": USB-C to RJ45 adapters never appeared (seen
2026-10-05, Dell USB-C adapter). igb lacked dca and i2c-algo-bit the
same way.

Walk the host modules.dep from the copied modules and copy every
dependency (73 on 6.8.0-71). Also copy rtl_nic, tigon, bnx2, bnx2x and
e100 firmware, which those drivers request at probe.
2026-10-05 11:56:16 -04:00
cproudlock
25f65424c7 blancco-init: load xhci_pci so USB-C NICs work, report why a link fails
The initrd has no udev, so only modules init names get loaded, and
xhci_pci is a module in the Ubuntu kernel. USB 3 / USB-C ports were
therefore dead in the UEFI Blancco path and a Dell USB-C to RJ45 adapter
never appeared; init took the onboard e1000e port with no cable.

- modprobe xhci_pci and xhci_pci_renesas before the NIC drivers
- bring up interfaces inside the carrier wait too, since USB NICs
  enumerate seconds after onboard ports
- on no-link, print each interface's driver and the NIC/firmware/link
  kernel messages before dropping to a shell

Live kexec-initrd.img also now carries the NIC firmware set from the
Blancco 7.20 early cpio (rtl_nic, tigon, bnx2x, ...) under /lib/firmware.
The playbook build does not add it yet.
2026-10-05 11:48:17 -04:00
cproudlock
29e0d97f59 blancco-init: use the cabled NIC, and stop hanging on a dead link
The UEFI init took the first interface in /sys/class/net and put
172.16.9.250 on it. On a PC with a second port, dock or USB NIC that can
be the unplugged one; wget then sat at "Connecting to 172.16.9.1:80"
forever (seen 2026-10-05 on a Dell, serial GGX6BN4). Bring every wired
interface up and use the first with carrier; with no link anywhere,
list the interfaces and drop to a shell. wget calls get -T 30 so a
stalled transfer fails loudly instead of hanging.

Live kexec-initrd.img was rebuilt with only init swapped (backup
kexec-initrd.img.bak-20261005-pre-carrier).
2026-10-05 11:38:57 -04:00
cproudlock
b474ac376e pxe: let BIOS Blancco boots load iwlwifi for cloud licensing
7.20 licenses through the Blancco cloud, and on the air-gapped PXE LAN
the only route out is WiFi INTERNETACCESS. The legacy BIOS cmdline still
blacklisted iwlwifi/iwlmvm, so BIOS boots could never license. btusb
stays blacklisted; rd.udev.timeout=10 bounds the udev hang iwlwifi
caused on some Intel machines. Live GetPxeScript.aspx changed to match
(backup .bak-20261005-pre-bios-wifi).
2026-10-05 10:14:57 -04:00
cproudlock
678d10d5c2 blancco: upgrade to Drive Eraser 7.20.0
playbook/blancco-preferences.xml is now the vendor 7.20 preferences
(schema 4.20, "Non_GEAE_Wired" profile) with only our site overrides:
SMB report share on 172.16.9.1 with auto_backup, WiFi INTERNETACCESS for
cloud licensing (wired stays up for SMB), manual as the fallback process,
bootable report, screensaver 86400. workflow default=true is kept: the
cloud workflow erases unattended, as live 7.15 already did. No custom
fields, matching both the vendor image and live 7.15.

prepare-boot-tools.sh:
- 7.20 initramfs is an uncompressed early cpio (NIC firmware, itself
  .xz files) followed by the XZ archive. The old patch decompressed from
  byte 0, failed, warned and carried on, so PXE boots would drop network
  after copytoram. Walk the cpio to its trailer and patch the XZ part.
- Patch exactly one copytoram match or fail. The looser fallback pattern
  matches five hooks in 7.20.
- config.img now carries playbook/blancco-preferences.xml, so the BIOS
  (native kernel) and UEFI (HTTP) paths read the same settings. Refuses
  to build if the playbook preferences are for another Blancco version.
- The old global replace() also set the 802.1x, WiFi and repeater
  passwords to "blancco". Gone with the above.
- Keep preferences.xml size by refilling the NUL gap before the trailing
  filler comment (7.20), or trimming the comment (7.15).
- OUT_DIR can be overridden to stage a build outside boot-tools/.
2026-10-05 09:32:32 -04:00
cproudlock
737a729548 fix(shopfloor): clear imaging AutoLogonCount at ShopFloor handover
Run-ShopfloorSetup and Stage-Dispatcher top AutoLogonCount up to 10 so
SupportUser survives vendor-forced reboots. Sysinternals Autologon, which
lockdown uses to switch the bay to ShopFloor, leaves the counter in place,
so ShopFloor inherited the remainder. Windows counts it down on every
autologon and at 0 sets AutoAdminLogon=0: the bay boots to a ShopFloor
password prompt a few reboots after imaging.

Delete the counter once Winlogon shows ShopFloor:
- Monitor-IntuneProgress.ps1, on the poll that sees the handover
- SetShopfloorAutoLogon.bat, after sfld_autologon.ps1
- Force-Lockdown.bat, on success

The SupportUser top-ups stay. Tested in the win11 VM: the Monitor block
removes the counter for ShopFloor, keeps it for SupportUser, and is a
no-op when clean; both batch files leave ShopFloor autologon on with no
counter.
2026-10-02 10:06:42 -04:00
cproudlock
f1755751bb Part marker: a misconfigured bay is now loud, and the listing loop is bounded
Every failure of Get-TelesisPortConfig was filed as "no Telesis marker
configured on this PC" and exited 0. That is correct for the many PCs which are
not marker bays, and wrong for a bay whose registry key EXISTS and could not be
read: a half-configured marker was silently never backed up again, which is
exactly the failure shape this feature was built to prevent. The absent-key case
now throws a distinguishable exception and stays quiet; anything else is logged
every cycle and counted as a skip.

Get-TelesisFileList was unbounded. The receiver drives that loop, so a
controller repeating a name or answering noise on a marginal cable would spin
for ever - and not harmlessly, because it holds the MARKING port while
GE-Enforce waits on the script with no timeout, so one chatty controller wedges
the enforcement cycle and the operator cannot mark a part. Now capped by count,
by elapsed time, and by repeat detection, all of which THROW: a partial listing
presented as a complete snapshot would be worse than failing.

Baud and Data Bits parse with TryParse. Note the narrower reason than it looks:
[int]'' is 0 in PowerShell, so empty and non-numeric values were already handled
by the zero-check. What was not handled is a numeric string too large for Int32,
which throws on the cast and would have been filed as "not a marker bay".

Verified in the win11 VM as SYSTEM, against both the old and new module: absent
key stays quiet, a key without Port Id is loud, garbage and out-of-range baud
both fall back to 9600, a real 19200 is still honoured, a repeating controller
stops at once and endless unique names stop at the cap.
2026-08-26 15:06:47 -04:00
cproudlock
8ba86ba560 Write through the mapped drive, record the UNC
The snapshot path stored against a marker read "W:\backup\partmarker\...". That
mapping exists only inside the SYSTEM logon session, only while the GE-Enforce
cycle is running, and only on that one bay - so the path recorded in ShopDB
resolved to nothing for the person most likely to need it.

Writing still goes through the drive letter, deliberately: that is the
connection GE-Enforce authenticated, and reaching the same server by another
name costs a second authentication or fails outright with error 1219. Only the
recorded path changes.

Resolving a mapped drive to its share needs all three of DisplayRoot, Root and
the WMI ProviderName, because which one is populated depends on how the drive
was made: net use mappings (what GE-Enforce creates) fill DisplayRoot, while a
New-PSDrive to a UNC leaves DisplayRoot EMPTY and puts the share in Root. A
local drive and an already-UNC path both fall through untouched.
2026-08-25 14:58:41 -04:00
cproudlock
f99ece1366 Make an unknown argument an error, not a shrug
The script had no CmdletBinding, so PowerShell dropped unrecognised named
arguments into $args instead of failing. A typo in a manifest entry Args -
"-WindowStartMinutes 45" - would have been ignored silently, and the capture
would have run at the wrong time with nothing in the log to explain it. That
is exactly how an hour went into debugging a time gate that was working.

Also adds WindowStartMinute so the window can open at :45 rather than on the
hour. A shift change is a moment, not an hour, and the first run of the day is
easiest to watch when it lands when somebody expects it.
2026-08-25 14:37:00 -04:00
cproudlock
ba4df185cd Put the marker entries where the share actually looks
Both entry files described the wrong scope. Checking the live share mirror:

Backup-PartMarker.ps1 belongs in common\scripts, not the partmarker scope,
because it imports ..\lib\TelesisMarker.psm1 and ..\lib\ShopdbBackupClient.psm1
and those live in common\lib. Backup-NtlarsSettings.ps1 sits there for the same
reason, and its entry already lists gea-shopfloor-partmarker in PCTypes -
scoping is by PCTypes, not by which directory the entry lives in. As written it
would have failed its imports on the first cycle.

eMxInfo healing is a NEW entry in the partmarker scope rather than a PCTypes
edit: those entries live in the collections and nocollections scopes, which a
marker bay never evaluates. The site already duplicates the config and the cmd
per scope, so markers follow that pattern - at the cost of the yearly rotation
now touching three copies, which the entry says out loud.

The cmd template also carried a stale source path. The deployed cmd reads
..\configs\eMxInfo.txt; the repo template still said the file sat beside it,
which would have exited 1 on every run.
2026-08-25 14:01:26 -04:00
cproudlock
bd131f6da9 Diagnose a share that prompts, when the same UNC path works
Shopfloor PCs intermittently lose S: and ask for credentials while navigating
the identical UNC path in Explorer works fine. Four different causes produce
that same appearance - a persistent reconnect that raced the network, a stored
credential whose target string does not cover the name form in use, two
identities contending for one server, or a password that rotated - and each
needs a different fix. From the desk they are indistinguishable.

Test-ShareAccess.ps1 collects what separates them: reachability of both name
forms, the mappings and the account each remembers, the exact credential target
strings, Kerberos tickets, live SMB sessions and the identities holding them,
and the SMBClient/Connectivity log. Then it names the causes the evidence
supports rather than leaving a wall of output to interpret.

Read-only by construction: it maps nothing, deletes nothing, stores no
credential, and prints usernames and a password-present flag but never a
password, because its log lands where techs read it. Safe on a bay while
somebody is working.

It reports which identity it is running as, because mappings and SMB sessions
are per logon session: run as the user to explain S:, as SYSTEM to explain W:
and the backup scripts. A drive missing in one is genuinely absent in the
other, and mistaking that for a fault is easy.

Also derives the marker backup's share root from the script's own location
rather than a hardcoded UNC. GE-Enforce mounts the share as W: with SFLD
credentials, and reaching the same server by a different name than the mount
used costs a second authentication - or fails outright with 1219 - for reasons
the error text never mentions. The script's own path is the one handle
guaranteed to name the connection that already exists.
2026-08-25 13:44:05 -04:00
cproudlock
18168b54f8 Say why a share write failed, because the symptom lies
This runs as SYSTEM, which authenticates to the network as the machine account.
A SYSTEM process reaching a UNC path without credentials gets an access-denied
that Test-Path reports as "not found", so a permissions problem reads as a
missing directory and sends the next person hunting the wrong thing. The
backups registry already carries that warning; the client had not acted on it.

On a write failure the log now names the likely cause, and the two conditions
that make it work: the entry must run under the GE-Enforce cycle, since the
share is mounted only for its duration and all SYSTEM processes share the logon
session that holds the connection, and the share ACL has to admit computer
accounts.

Verified against an unreachable UNC on a bay with a real controller: capture
succeeded, the write failed, the diagnostic was logged and the cycle was
counted as a missed backup.
2026-08-25 13:32:23 -04:00
cproudlock
b228a69364 Identify a marker by its device, without moving what NTLARS reports
A part marker has no machine number of its own - the device is what a backup
belongs to, and C:\Enrollment\asset-id.txt is what names it. Reading that file
inside the shared machine-number lookup looked like the tidy place for it and
was wrong: marker bays run eDNC too, so Backup-NtlarsSettings.ps1 reaches the
same lookup, and it would have started posting a machine's DNC settings under
the marker's asset number where today it exits quietly. The marker script asks
for the fallback explicitly instead, which is what -MachineNumberFallback is
for.

Also from review:

Returning a byte array bare lets the pipeline enumerate it, so an EMPTY file
came back as $null, ComputeHash threw, and with ErrorActionPreference=Continue
the manifest entry silently kept the previous file's hash - a corrupted
manifest stored as a good revision. Empty parameter files are plausible.

Reading registry values directly under StrictMode throws when one is absent, so
the "no Port Id" guard was unreachable and a partially configured marker bay
reported itself as having no marker at all, permanently and quietly. Values are
now read through a presence check with the fleet's own defaults.

The outside-window quiet state embedded the hour, so the say-once module said
it hourly. Command replies now have their checksum verified like data blocks
do, since a corrupted listing entry would otherwise be trusted and a file
requested under a name the controller does not have. A controller that
genuinely holds nothing is no longer reported as a failed capture. A snapshot
that could not be written to the share now counts as a missed backup like the
rest, and the header no longer claims orphaned snapshots get tidied - they are
kept on purpose, because a timed-out post may still have been recorded.
2026-08-25 13:21:45 -04:00
cproudlock
abbdc9873c Heal eMxInfo.txt on marker bays too
The eMxInfo entry on the share is scoped to collections and nocollections, so
a marker bay never heals. Marker bays run eDNC as well - 01-eDNC.ps1 installs
it before the Mark MSI - so the same vendor installer clobbers the same file
there, with nothing to restore it.

The fix is a scoping change rather than a new mechanism: add the marker PCType
to the existing entry. The paths it already writes are the right ones, because
mxTransactionDll.dll in the Mark overlay reads \Dnc\Server Files\eMxInfo.txt -
the Mark suite uses the DNC install, not its own directory.

The cmd variant additionally refreshes the copy that imaging drops into the
Mark directory. Nothing reads that path, so it is cosmetic; it is there so a
healed bay and a freshly imaged one are identical, rather than differing in a
way that costs somebody an afternoon later.
2026-08-25 13:10:29 -04:00
cproudlock
280a3b3ba4 Add the manifest entry that puts the marker backup on the fleet
Scoped to gea-shopfloor-partmarker so it reaches marker bays only. The script
checks the DNC\Mark registry itself as well, so a mis-scoped entry exits
quietly on an ordinary PC rather than erroring every five minutes.

The entry records what the share layout has to provide, because the script
resolves its two modules relative to its own location and one of them,
ShopdbBackupClient.psm1, is shared with the NTLARS backup and has changed. It
also records the schedule and the settings a bay needs, since all three are
places where a deployment can look successful and quietly do nothing.

Also counts a failed capture as a missed backup, not just a busy port. Port
contention resolves itself; a controller that will not answer does not, and
the two are worth telling apart in the count that reaches ShopDB.
2026-08-25 13:09:17 -04:00
cproudlock
e40a294193 Back up what a part marker holds, without the vendor tool
Telesis pinstamp controllers have never been backed up. The vendor's own
utilities, Backup470 and Backup420, are GUIs with no command line, so nothing
could capture a marker on a schedule - and a controller swap meant retyping
patterns from memory.

TelesisMarker.psm1 speaks the protocol directly, reversed from a capture of
Backup470 against a TMC470 and verified against hardware: a pattern pulled by
this code is byte-identical to the one the vendor tool saves, and a pattern
written back survives a power cycle. A TMC420 was captured the same way and
differs in exactly one respect - it exposes no parameter files - so one
implementation covers both families, with content classes chosen from a Type I
status probe rather than a per-model dialect.

Backup-PartMarker.ps1 captures everything the controller holds into a
timestamped snapshot on the share and records it in ShopDB against the marker
asset. Snapshots rather than a mirrored directory, because a mirror forces a
choice between deleting the only copy of a retired pattern and claiming a
pattern that no longer exists; a snapshot has neither problem, and the manifest
hash means an unchanged controller writes nothing.

The marker's serial port is shared with eDNC and is exclusive-open, so captures
happen only at shift changes - 14:00 normally, when someone is on site to fix a
problem, with 06:00 and 22:00 engaging only once the backup is overdue. The
capture yields the port within a second if the marker application starts
mid-run, and cycles missed to a busy port are counted and carried into the next
snapshot, since posting a no-op to keep the health badge green would hide
exactly the failure the badge exists to report.

TYPE 'R' deletes a file on the controller, immediately and without
confirmation. It is documented in the module header and deliberately not
implemented.
2026-08-25 13:02:06 -04:00
cproudlock
12c6c771dc Stage the two universal print drivers at imaging time
A bay comes off the line unable to build a printer queue. Set-ShopdbPrinters.ps1
creates queues against drivers that are already present and skips any that are
not, so a freshly imaged PC silently gets no printers until some later
enforcement cycle tops it up.

The two universals are 100 MB and resolve the overwhelming majority of the
fleet's printers. The four device-specific drivers are another 126 MB and are
deliberately NOT here - they belong only on bays that own that hardware, and
those bays pick them up from the runtime manifest.

NO WAVE GATE on these entries, unlike the runtime ones. Waves exist because ~300
bays share one runtime share and GE-Enforce spreads them across only five
minutes; imaging runs a handful of bays at a time on the isolated PXE LAN that
already moves multi-GB WIMs, so there is no stampede to prevent.

DETECTION IS THE SPOOLER'S OWN KEY, not pnputil, and the difference is not
cosmetic. Installing a printer driver is two steps: pnputil /add-driver stages
the package into the driver store, Add-PrinterDriver registers it with the
spooler. Get-PrinterDriver - which is what Set-ShopdbPrinters consults - reads
the spooler. Measured on a Win11 bay: after Remove-PrinterDriver the spooler
reports the driver gone while pnputil still lists the package, so a pnputil-
detected entry would report compliant forever while every queue kept skipping. A
broken bay that looks healthy is the worst outcome available here.

The payload is not in git, matching every other installer under preinstall/:
stage printer-drivers/ onto the PXE share alongside the rest.
2026-08-19 17:16:38 -04:00
cproudlock
d479e6823c Clear comments orphaned by today's unattend removals
The EAP-PEAP and Office comments outlived the entries they labelled, which
would send the next reader looking for commands that are not there. The
surviving wireless comment said 'Wi-Fi profiles' plural when only
INTERNETACCESS remains; it now records why the other two went and that
wlansvc has to be started first.
2026-08-19 16:07:37 -04:00