Well-formed XML is not enough. Windows Setup validates against a schema and one
bad value invalidates the WHOLE answer file for its pass, so the machine stops at
a dialog with nothing configured and the only clue is an XPath buried in
C:\Windows\Panther\setupact.log.
On 2026-08-06 every shopfloor, standard and engineer build was failing exactly
that way:
/settings/RunSynchronous/RunSynchronousCommand/[Order="16"]/Path
Description = Value is invalid. hrResult = 0x80220005 pass = specialize
An inlined "powershell.exe -Command ..." had grown to 676 characters in a field
capped at 259, and had been broken since the previous evening. Every check here
is mechanical and would have caught it before a bay was booted:
path-too-long RunSynchronousCommand/Path > 259
cmdline-too-long SynchronousCommand/CommandLine > 1024
description-too-long Description > 256
duplicate-element a once-only element appearing twice
not-well-formed XML does not parse
bom UTF-8 BOM (the live files have none, and python's
utf-8-sig silently ADDS one when writing them back)
unknown-token %token% PESetup will not substitute; %WINDIR% and the
other shell variables are excluded or it cries wolf
Reads over SSH via base64 so a BOM or CRLF survives the hop unchanged. Exits
non-zero on any ERROR so it can gate a deploy.
Immediately found a second live defect the manual review had missed: a 329-char
Description on gea-engineer Order 11, which would have failed the oobeSystem pass
once specialize started passing. All three live image types and both repo copies
now lint clean.
CopyPackages copies the whole of <media>\Deploy\Applications to the target,
recursively, with no manifest and no filtering, so every bay built from the
shared gea-shopfloor media receives every shopfloor payload whatever its type.
The obvious fix - repoint a path inside the shared media as each bay picks its
type - races. Bays image concurrently and CopyPackages is fail-fast, so moving a
path while another bay is mid-copy either hands it the wrong payload or fails its
imaging outright, intermittently and unreproducibly.
Give each type its own media directory instead, built once and never mutated.
Everything in it is a symlink, so a type costs inodes rather than gigabytes and
the shared media stays untouched while bays read it. Samba already serves this
pattern: follow symlinks and wide links are on, and every image root reaches
Sources, Operating Systems, Out-of-box Drivers and Packages the same way.
Types come from the enrollment share's menu.json, the file the WinPE picker
already renders, so the media set cannot drift from the boot menu. Deliberately
NOT mirrored: the FlatUnattendW10.xml.pre-* backups and stray logs beside the
real unattend - the view is the clean set PESetup actually reads. Each type is
built into a scratch dir and swapped, so a bay mapping the path mid-refresh sees
the old tree or the new one, never a half-built one.
Applied on 172.16.9.1: 10 types, 190 links, all 60 critical paths resolve. An
SMB client sees ordinary directories and files - Sources, Control,
FlatUnattendW10.xml at 18078 bytes, HardwareDriver.json at 46113 - which is the
behaviour that actually matters, since PESetup reads this over SMB from WinPE.
Note for anyone extending this: passing the server-side script on stdin to
"echo pxe | sudo -S bash -s" is a silent no-op. Bash inherits the password pipe,
reads EOF, runs nothing and exits 0. It travels as a heredoc for that reason.
A driver miss is only a warning: GetDriverByModel returns null, PESetup logs
"driver for [MODEL] not found" and images the machine anyway. The bay comes up
with no NIC and no WiFi, DNS fails at first boot, and bulk enrollment cannot
reach the CDN - symptoms far enough from the cause that the OptiPlex Micro 7020
pack sat missing and the Display MicroPC failures were blamed on a drive letter.
Reimplements the matcher from the decompiled source (docs/PESETUP-INTERNALS.md)
and reports what silently breaks it:
virtual-platform one such entry flips the tool into virtual-only mode and
hard-fails every physical machine
empty-token a trailing comma yields "", and Contains("") is true for
every model, so that entry swallows the catalogue
token-whitespace Split(',') does not trim, so " OptiPlex 3010" needs the
space present in the model string too
case-mismatch the share is case-sensitive; Optiplex vs OptiPlex splits the
tree and the pack is never found
missing-zip referenced pack absent
family-mismatch the family filter runs first, so a token whose line
contradicts the family field can never match
shadowed first match wins, so a later entry may be unreachable
duplicate-token osId is not part of the match, so a win10 pack can land on a
win11 build purely by ordering
--models resolves real WMI model strings through the same code, which is the
check that actually predicts a no-driver build. Exits non-zero on ERROR or
CRITICAL so it can gate a deploy.
Verified both ways: a synthetic catalogue carrying each defect reports all eight
and exits 1; the three live catalogues on 172.16.9.1 come back clean at 0. The
file listing needs find -L and the same anchoring as destinationDir - without
either, every zip check silently passes.
parse_bios_catalog kept the 'latest' BIOS per model with a string compare, so e.g. '1.20.1' > '1.9.0' was False and it wrongly retained the older 1.9.0. Added _ver_tuple() and compare tuples of ints so the genuinely newest firmware wins.
Single-site bay-stuck issue at WJ: GE Intune Report IP script filters
Get-NetIPAddress on StartsWith("10.") and posts everything matching
to the GE Tines webhook. Bays at WJ get the PXE LAN 10.9.100.x IP
captured and reported -> GE backend tags bays as on a non-corp 10.x
subnet -> dynamic group eligibility for SFLD policy never matches.
Other GE sites work because their PXE LANs aren't on 10.x at all.
Renumber PXE LAN to RFC1918 172.16.9.0/24 so the GE filter naturally
skips wired PXE addresses without any disable-NIC dance.
Server-side already in flight (netplan dual-bound, dnsmasq scope +
boot URL repointed, blancco preferences + grub.cfg + iPXE GetPxeScript
all sed'd to 172.16.9.1). This commit is the playbook / scripts /
docs side: 109 hits across 35 files sed'd in one shot.
After this lands + boot.wim is rebuilt + bays renumber off DHCP,
the 10.9.100.1 binding will be dropped from netplan as the final
cleanup step.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three related fixes:
1. Hard-coded BIOS push path was /srv/samba/enrollment/BIOS, which does
not exist on the live PXE server. Real path is the shared
/srv/samba/winpeapps/_shared/BIOS/ where check-bios.cmd lives and
playbook task pxe_server_setup.yml:485 deploys Flash64W.exe + the
per-model BIOS .exe files.
2. Generated models.txt was 2-column (ModelSubstring|BIOSFile) but
check-bios.cmd reads tokens=1,2,3 with delims=| and uses field 3
for the version compare. Without the 3rd column, the version-skip
logic never engages and every imaged PC re-flashes BIOS on every
boot. Now writes 3-column (ModelSubstring|BIOSFile|Version).
3. The script overwrote the live models.txt with only the entries it
touched in the current run. Live had 50+ entries; a single-model
run wiped the other 49. Now prints the lines and asks the operator
to merge them into playbook/shopfloor-setup/BIOS/models.txt and
re-deploy via scripts/deploy-bios.sh.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
scripts/diagnostics/Capture-LockdownState.ps1 captures Windows endpoint
state at three lifecycle checkpoints so the deltas isolate which phase
delivered (or failed to deliver) each component:
- pre-category - PPKG-enrolled, no Intune category yet
- post-category - category-driven assignments arrived, pre-lockdown
- post-lockdown - kiosk + autologon + AppLocker fully landed
Bumped from the previous 2-stage (pre/post) version. Legacy 'pre'/'post'
aliases preserved.
Captures additions driven by the SFLD-DSC v2.0.2 post-mortem:
- IMECache file listing (catches missing sastoken.txt)
- DSCDeployment.log + version.txt copied from C:\pc\
- SFLD\DSC payload listing
- C:\Logs\BPRT\ runtime state (criticalChecks.json, packageInfo.json)
- C:\WCDApps\ deploy verification
- Windows\Provisioning\Diagnostics copy
- Tasks-RunHistory.csv with LastRunTime + LastTaskResult per task
- DeviceManagement-Events.csv (MDM 429s, AAD token failures)
- Provisioning-Events.csv (PPKG runtime errors)
- MDM-Certificates.csv (enrollment cert health)
scripts/diagnostics/snapshot-runbook.txt: step-by-step ops guide
covering when to fire each stage, where output lands, how to ship it
back via image-upload share, and which files to compare first when
diffing.
Shebang was \`#!/usr/bin/env python3\r\` which errored with
\`/usr/bin/env: python3\r: No such file or directory\`. Pure line-ending
fix, no content change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Consolidates per-type enforcers (CMM, Keyence, Machine, Common, Acrobat)
into one dispatcher driven by pc-type.txt + site-config and a share-side
manifest layout. Same share is now the single source of truth for routine
software updates without re-imaging.
Runtime:
common/GE-Enforce.ps1 SYSTEM scheduled task. Reads
common/manifest.json plus optional
<pcType>/manifest.json and
<pcType-subType>/manifest.json.
Dispatches each entry through the lib.
Writes _outputs/logs/<hostname>/status.json
on the share after each cycle for fleet
monitoring.
common/Register-GEEnforce.ps1 Task registration. Triggers: AtLogOn +
every 5 min (jittered per-PC from
hostname hash) + daily at 05:45,
13:45, 21:45 EST shift windows.
Unregisters legacy per-type tasks on
install so the two coexist at most for
the duration of a single enforce cycle.
common/Deploy-GEEnforce.ps1 Retrofit helper for already-imaged PCs
(admin-run; copies runtime + registers
task + optional immediate trigger).
Library (common/lib/Install-FromManifest.ps1):
- New Type values: PS1, BAT, File, Registry, INF
- New DetectionMethod values: Always, MarkerFile, ValueMatches, pnputil
- TargetHostnames filter (exact + -like wildcards, ANDed with PCTypes)
- Schema version check (logs WARN on manifest newer than lib MAJOR)
- Auto-writes MarkerFile on successful one-shot PS1/BAT/CMD runs
- MSI log scan on failure surfaces meaningful install errors
- Lib version bumped 2.0 -> 2.1 for TargetHostnames
Observability:
common/monitor-fleet-status.py Scans _outputs/logs/*/status.json for
stale check-ins, failed scopes, and
version drift. Respects scope (dir-name),
PCTypes, and TargetHostnames filters so
entries excluded from a PC do not
false-flag as drift.
Regression harness:
common/test/ Parameterized VM harness + README
covering every action type plus
rollback, bad/missing SFLD creds, and
schema versioning.
Imaging integration:
Run-ShopfloorSetup.ps1 now stages GE-Enforce.ps1 and lib to
C:\Program Files\GE\Shopfloor\ and invokes Register-GEEnforce.ps1
at the end of setup. Legacy Register-CommonEnforce invocation is
kept for the transition; it and the legacy per-type enforcer files
are dead code once Register-GEEnforce runs and will be removed in a
dedicated cleanup pass.
Standard-Machine manifest:
eDNC entry bumped 6.4.3 -> 6.4.5. DetectionValue pinned to the
4-part FileVersion 6.4.5.0 verified against a fresh install in the
Win11 analyzer VM. UDC DetectionValue pinned to 1.0.34 (registry
stores 3-part for UDC; verified live).
scripts/mirror-from-gold.sh:
Restructured around share-root rsyncs (one pass per Samba share)
to close gaps in the prior per-subdir layout: winpeapps/_shared/
Applications (7.5 GB of Adobe + fonts + Java + Office + OpenText
+ printdrivers + wireless + Zscaler), additional winpeapps image
types, and enrollment flat-layout root files. Adds
--skip-clonezilla and --skip-reports.
Verified end-to-end in the Win11 analyzer VM:
- Every action Type and DetectionMethod round-tripped
- PCTypes filter (Oracle excluded on Shopfloor, Firefox included
on Shopfloor and DESKTOP-*, excluded elsewhere)
- TargetHostnames filter (exact, wildcard, no-match)
- Upgrade path: XML hash bump + fleet re-copy
- Rollback path: history-archive restore propagates via enforcer,
fleet converges back without per-PC intervention
- Status writeback + monitor script drift detection
- Graceful degradation on bad creds, missing creds, share
unreachable (all exit 0, log clearly, retry next cycle)
Not in this commit (follow-ups):
- Retire legacy per-type *-Enforce.ps1 files and simplify
09-Setup-*.ps1 scripts (coordinated multi-file cleanup)
- Stage 2b: InUseCheck close-and-reopen, ApplyMode gating,
UpdateWindow, .apply-now.txt sentinel, BITS pre-staging,
1618 mutex retry, PostInstallCheck, Uninstall action
- Management app (manifest CRUD + deploy + rollback + fleet view)
- ShopFloor autologon persistence bug (deferred for next imaging
attempt with live registry evidence)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- playbook/startnet.cmd + startnet-template.cmd: after preinstall staging,
xcopy Y:\pre-install\udc-backups to W:\PreInstall\udc-backups so UDC
settings JSONs are available during image deployment. Harvested from
live gold where this block existed but was never committed.
- scripts/mirror-from-gold.sh: update source paths to current taxonomy
layout (pre-install/, installers-post/, blancco/, config/) and add
ppkgs/, scripts/, shopfloor-setup/ sections. Added --delete for exact
mirror semantics. Used to seed the spare PXE server at 10.9.100.2 on
2026-04-16 from gold at 10.9.100.1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two new docs cover what the system is (boot chain, services, shares,
enrollment layout, data flow) and what to change per site (every
hardcoded value, where it lives, secrets handling).
scripts/mirror-from-gold.sh replicates content from an existing PXE
server (Operating Systems, drivers, packages, custom installers, BIOS,
PCDMIS, Blancco custom image, site-config) onto a freshly-installed PXE
server. Translates the legacy flat enrollment layout on the source into
the reorganized taxonomy (ppkgs/, pre-install/installers/, installers-
post/cmm/, blancco/, config/) on the destination. Tolerates rsync
exit 23 (permission-denied subdirs like the OpenText W10shortcuts dir
that is pxe-upload-group-only on legacy servers).
Workstation reorganization:
- All build/deploy/helper scripts moved into scripts/ (paths updated to use
REPO_ROOT instead of SCRIPT_DIR so they resolve sibling dirs from the new
depth)
- New config/ directory placeholder for site-specific overrides
- Removed stale: mok-keys/, test-vm.sh, test-lab.sh, setup-guide-original.txt,
unattend/ (duplicate of moved playbook/FlatUnattendW10.xml)
- README.md and SETUP.md structure listings updated, dead "Testing with KVM"
section removed
- .claude/ gitignored
Enrollment share internal taxonomy (forward-looking; existing servers
unaffected since they keep their current boot.wim with flat paths):
- Single SMB share kept (WinPE only mounts one Y: drive), but content now
organised into ppkgs/, scripts/, config/, shopfloor-setup/, pre-install/{bios,
installers}, installers-post/cmm/, blancco/, logs/
- README.md deployed to share root explaining each subdir
- New playbook tasks deploy site-config.json + wait-for-internet.ps1 +
migrate-to-wifi.ps1 explicitly (were ad-hoc on legacy servers)
- BIOS subdir moved into pre-install/bios/, preinstall/ renamed to pre-install/
- startnet.cmd + startnet-template.cmd updated with new Y:\subdir\ paths
- Bumped GCCH PPKG references v4.9 -> v4.10
Blancco USB-build fixes (so next fresh USB install boots Blancco end-to-end
without the manual fixup we did against GOLD):
- grub-blancco.cfg: kernel/initrd switched HTTP -> TFTP (GRUB's HTTP module
times out on multi-MB files); added modprobe.blacklist=iwlwifi,iwlmvm,btusb
(WiFi drivers hang udev on Intel business PCs)
- grubx64.efi rebuilt from updated cfg
- Playbook task added to create /srv/tftp/blancco/ symlinks pointing at the
HTTP-served binaries
run-enrollment.ps1: OOBEComplete is now set AFTER PPKG install (Win11 22H2+
hangs indefinitely if OOBEComplete is set before the bulk-enrollment PPKG runs).
Also includes deploy-bios.sh / pull-bios.sh / busybox-static / models.txt
that were sitting untracked at the repo root.
