Rebuilt the /startnet editor (Fable/Opus 4-stage build) into tabs:
- Settings: server IP (+ menu timeout/default when a choice construct exists) as form fields; apply rewrites only the targeted tokens.
- Boot Menu: add/remove/reorder image entries; regenerates only the menu echo/dispatch + action blocks, refuses reorders that would desync the enrollment %choice% router.
- Raw: full-text editor (still source of truth) with line-number gutter, batch syntax highlighting, a lint panel (unmatched goto/label, CRLF), and diff-vs-current.
- History: timestamped snapshots on every save, per-row diff + restore.
New wim.py helpers (framework-free): parse/apply_settings, parse/apply_boot_menu, lint_startnet, save/list/read_snapshot, unified_diff; BACKUPS_DIR=/var/lib/pxe-webapp/startnet-backups.
Fable review fixed a CRITICAL pre-existing bug: update_startnet's newline=CRLF write retranslated posted CRLF into \r\r\n, corrupting boot.wim on every raw save; now normalizes to LF first. Also fixed a false CRLF lint warning (verbatim read) and menu payload validation. All JS inline (no CDN).
startnet.cmd now sources SOURCE_PPKG/PPKG_VER/PPKG_EXP from \\<pxe>\enrollment\ppkg.conf (baked-in values kept as fallback if the file is missing). The webapp writes ppkg.conf on upload for standard GCCH_..._v<ver>.ppkg files: version parsed from the filename, optional expiry field (blank keeps current), so a new ppkg goes live at next boot with no startnet/boot.wim edit. Enrollment page shows the active ppkg. ppkg.conf seeded on the share (v4.16) and the ppkg.conf-aware startnet.cmd deployed into boot.wim.
Image import previously full-copied everything and rmtree'd existing target dirs on every run. Replaced the shutil copy/move/copytree/rmtree with deploy.sync_tree(), which shells out to rsync -a --checksum: files whose content already matches the target are skipped, only new or changed files are written, and existing target files not in the source are left untouched (merge, not mirror). move=True uses --remove-source-files (frees the SMB upload dir) and prunes emptied source dirs. Applies to the Deploy import, the _shared redirections, and the root-level items. Big re-imports now only rewrite what actually changed.
Imaging dashboard
- services/imaging_log_tail.py: parses dnsmasq leases, Apache access log,
Samba per-host log files, and dnsmasq syslog (DHCP/TFTP). Synthesizes
inferred sessions keyed by MAC for bays that have only touched the boot
chain but not yet pushed to /imaging/status. Active window 90 min.
- imaging_status.list_sessions() merges inferred sessions into the dashboard
list. Real client-pushed sessions win for the same MAC.
- imaging_status: stage_history field tracks every stage transition (capped
30); sidecar .log file per serial records every log_lines push uncapped
(read_full_log() caps detail-page response to 1 MB).
- delete_session/delete_all_sessions clean up sidecar .log too.
- New SSE endpoint /imaging/stream emits a session-list hash every 5s.
Client fetches /imaging/tiles (HTML partial) on hash change and swaps
#imaging-tiles innerHTML. Polling fallback at 15s if SSE drops.
- Tile-swap preserves scroll, filter input, expanded state via localStorage,
and any LAPS input the operator is mid-pasting (swap skipped when a
laps-input is focused).
- imaging.html: removed 15s location.reload(). Added live-status dot in
header (gray idle / green SSE connected / red SSE lost).
- _imaging_tiles.html: shared partial used by both /imaging full render and
/imaging/tiles SSE refresh. Inferred bays render with yellow border +
log-inferred badge + no progress bar (stage_index inference is coarse).
- imaging_detail.html (new): per-bay forensics page at /imaging/session/
<serial>. Session metadata grid, stage timeline table, full sidecar log
with truncation indicator, Copy-support-summary button. Linked from each
client-pushed tile.
- qr-render.js exposes window.renderAllQRs() so the SSE swap can re-render
Intune device-ID QRs in the swapped-in tiles.
Image management
- services/image_registry.py: JSON registry of image types at
{SAMBA_SHARE}/image-registry.json. Bootstraps from baked-in
config.IMAGE_TYPES on first run. create/clone/delete/rename_friendly
mutate the file then call reload() which rewrites config.IMAGE_TYPES +
config.FRIENDLY_NAMES in place. Sidebar reflects on next request.
- app.py routes: /images/new, /images/<t>/clone, /images/<t>/delete (with
optional content-wipe checkbox), /images/<t>/rename.
- dashboard.html: + New image type button + Clone/Delete per row, all in
Bootstrap modals with confirmation copy.
- Clone copies Deploy/ tree but preserves symlinks to shared dirs (Out-of-
box Drivers, Operating Systems, Packages) so disk usage stays low.
- Delete with content checked unlinks symlinks (does not follow into shared
dirs).
Driver / package upload + orphan adoption
- services/images.py: upload_driver, adopt_orphan, remove_orphans,
upload_package. Filename sanitization blocks path traversal.
- app.py routes: /images/<t>/drivers/upload, /images/<t>/drivers/adopt,
/images/<t>/drivers/orphans/delete, /images/<t>/packages/upload.
- image_config.html: Upload .zip button + modal on Drivers section. Orphan
drivers card-footer rebuilt as interactive list with per-row Adopt inline
form (family + destinationDir inputs) and bulk select+delete.
- Upload .zip on Packages section with optional destinationDir field that
appends a packages.json entry.
Configuration
- config.py: new env vars DNSMASQ_LEASES, APACHE_ACCESS_LOG, SAMBA_LOG_DIR,
DNSMASQ_SYSLOG for the log-tailer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New /imaging/delete_all endpoint wipes every per-bay JSON in IMAGING_DIR
via imaging_status.delete_all_sessions(). Template adds "Clear all"
outline-danger button next to the count badge, gated on sessions list
non-empty, with confirm() prompt naming the count.
Deployed via scp + systemctl restart pxe-webapp on 172.16.9.1.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous /imaging/<serial>/laps clear path used update_session() to
re-feed state minus laps_password. But update_session MERGES payload
into existing state - it cannot delete a key the existing state
already has. The laps_password persisted on disk across the "clear"
POST, then came back into the page on next reload.
Fix: bypass update_session for the clear case. Read the session JSON
directly, pop laps_password, write via atomic tempfile-rename. Same
write pattern update_session uses for consistency.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1. Phase 1 done gate was requiring 'AESFMA WLAN connected' in addition
to the data-side signals (AAD + Intune + EmTask + baseline). If the
bay never reached AESFMA (cert never landed, RADIUS unreachable),
Phase 1 stayed IN PROGRESS forever even though Intune registration
was actually complete. Reverting to the data-side-only definition.
2. New webapp endpoint POST /imaging/<serial>/laps stores a LAPS
password in the session JSON so it survives the 5s dashboard
auto-refresh. Empty body clears the field. Daily reset of the
server (cron/restart) is the lifetime cap on stored passwords.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Drops the filename, type, and size columns from the Blancco Reports
list - operators want bay-identification fields, not file metadata.
Filename moves to a row hover tooltip (title attribute) so it is still
recoverable for ad-hoc lookups.
Adds a Result column derived from each XML report's overall erasure
state:
* Successful -> green badge (all erasure entries report Successful)
* Failed -> red badge (any erasure entry reports a non-Successful state)
* other -> grey badge with the verbatim state
* blank/non-XML -> dash
The state roll-up lives in the blancco_reports route's per-file parse
loop next to the existing serial/model extraction.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds end-to-end progress tracking for PXE imaging sessions and surfaces
each Blancco report's BIOS serial in the report list.
webapp:
* services/imaging_status.py - JSON-per-serial state store under
IMAGING_DIR (default /var/log/pxe-imaging). Atomic write via
tempfile + rename. log_tail capped at 50 lines. Merges partial
updates so clients can post just the current_stage tick.
* config.py - new IMAGING_DIR env-overridable path.
* services/csrf.py - explicit exempt list for machine-to-machine
endpoints; /imaging/status is the first entry. Air-gapped LAN;
trust-by-network for client posts.
* app.py - four new routes:
GET /imaging dashboard (renders all sessions)
POST /imaging/status client status push (JSON body)
GET /imaging/<serial>.json raw session JSON for ad-hoc polling
POST /imaging/delete/<s> clear a session from the dashboard
Also parses each Blancco XML in the /reports list to surface
system.serial + system.model columns.
* templates/imaging.html - Bootstrap dashboard with per-session
cards (state badge, progress bar, stage idx/total, mac, elapsed,
log tail). meta http-equiv refresh=5 for auto-tick.
* templates/base.html - new "Imaging Progress" nav entry.
* templates/reports.html - Serial + Model columns added.
playbook:
* shopfloor-setup/Shopfloor/lib/Send-PxeStatus.ps1 - new helper.
Dot-source this then call Send-PxeStatus -Stage X -StageIndex N
-StageTotal M from any stage script. BIOS serial via CIM, MAC via
Get-NetAdapter, pctype + machinenumber from C:\Enrollment.
Failures are swallowed to a local log so a network blip doesn't
block imaging.
* shopfloor-setup/Run-ShopfloorSetup.ps1 - dot-sources helper +
posts at three coarse milestones (start, PPKG enrollment,
handoff to Monitor-IntuneProgress).
* shopfloor-setup/gea-shopfloor-keyence/09-Setup-Keyence.ps1 -
posts at session start + after Install-FromManifest with
succeeded/failed status derived from $rc. Other 09-Setup-*.ps1
scripts can follow the same pattern.
ID is BIOS serial (stable across WinPE -> Windows transition and
across reboots, unlike hostname which is random pre-PPKG). Operator
already knows the serial of the bay they imaged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
End-to-end fixes for Blancco Drive Eraser PXE flow uncovered by chasing
"reports never reach SMB share" across two air-gapped sites:
playbook/blancco-init.sh:
* Drop silent || true on wget of preferences.xml + config.xml. Fail
loud with shell-drop if download or marker grep fails. Background:
airootfs /opt/scripts/validate_preferences.sh restores
/albus/preferences.save (factory defaults, empty network_share) if
xmllint fails. wget failure made every report silently land nowhere.
* Clobber /albus/preferences.save with the same served file so even if
the validator fallback fires, the SMB target survives.
* Bind-mount /dev/null over /sys/power/{state,disk,mem_sleep,autosleep}
before switch_root. Albus's license-retry path writes /sys/power/state
directly (bypassing systemd targets); this is the last-line block.
* /dev/null symlinks for sleep/suspend/hibernate systemd targets in the
airootfs overlay + logind drop-in with IdleAction/Handle*=ignore.
Three independent layers because cmdline systemd.mask alone is bypassed
by direct /sys/power/state writes.
* xinitrc.d/00-no-screen-blank.sh runs xset s off -dpms + setterm
-blank 0 -powerdown 0 so the Blancco GUI doesn't blank during long
erasures.
* Removed the 20-failsafeDriver.conf "modesetting" pin. modesetting
needs DRM/KMS which we disable on kernel cmdline; "vesa" also failed
on NVIDIA. With the pin gone Xorg auto-picks fbdev which uses the
kernel framebuffer from vga=normal - works across Intel, AMD, and
older NVIDIA without nouveau.
playbook/pxe_server_setup.yml:
* dnsmasq.conf: explicit empty-value dhcp-option=3 + dhcp-option=6.
Without them, dnsmasq defaults to sending its own IP as router AND
DNS. Commenting the configured-value lines did NOT disable the push
(root cause of "wired keeps picking up 10.9.100.1 as gateway").
* Split the Blancco config.img extraction and preferences.xml deploy
into separate tasks. The previous shell-with-creates: gate caused
playbook re-runs to skip the prefs deploy entirely after first run.
* Added a validation task that runs python3 xml parse + grep on the
deployed preferences.xml to fail the playbook at deploy time if the
SMB markers are missing.
* Added Environment=TZ=America/New_York to the pxe-webapp systemd
service so report mtimes and audit log render in Eastern time even
if the Python process is started before timedatectl converges.
webapp:
* services/blancco_report.py: parse Blancco's XML report format
(recursive <entries name="..."> walker) into a friendly dict.
* templates/report_view.html: Bootstrap "Drive Erasure Certificate"
layout - hero summary, customer + system cards, per-drive cards with
step-by-step erasure timeline, document signing footer with
integrity hash detail.
* /reports/view/<filename> route + View button on the reports list
(XML reports only; PDFs still download).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The playbook deploys SFLD provisioning packages to
/srv/samba/enrollment/ppkgs/ but the /enrollment route scanned the
share root. Result: every visit reported "no enrollment packages
found" even though three .ppkg files were present.
Add ENROLLMENT_PPKG_DIR (defaults to {ENROLLMENT_SHARE}/ppkgs,
overridable via env var) and point all four enrollment routes at it
(list, upload, download, delete).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Webapp now creates Deploy/Control/Media.tag after every image import.
Cron updated to create (not just touch) Media.tag for any image
directory that has Deploy/Control/.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add startnet.cmd: FlatSetupLoader.exe + Boot.tag/Media.tag eliminates
physical USB requirement for WinPE PXE deployment
- Add Upload-Image.ps1: PowerShell script to robocopy MCL cached images
to PXE server via SMB (Deploy, Tools, Sources)
- Add gea-shopfloor-mce image type across playbook, webapp, startnet
- Change webapp import to move (not copy) for upload sources to save disk
- Add Samba symlink following config for shared image directories
- Add Media.tag creation task in playbook for drive detection
- Update prepare-boot-tools.sh with Blancco config/initramfs patching
- Add grub-efi-amd64-bin to download-packages.sh
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Playbook: detect interface already configured with 10.9.100.1 before
falling back to non-default-gateway heuristic (fixes dnsmasq binding
to wrong NIC when multiple interfaces exist)
- test-vm.sh: auto-attach br-pxe bridge NIC if available on host
- Webapp: add network upload import via SMB share with shared driver
deduplication and symlinks
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Move Flask to localhost:9010, Apache serves port 9009 with static file
handling and reverse proxy to fix intermittent asset loading on remote clients
- Add "PXE Manager" branding beneath logo in sidebar
- Increase code editor size (startnet.cmd and unattend XML) to 70vh
- Add test-lab.sh for full lab VM testing
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add build-proxmox-iso.sh: remaster Ubuntu ISO with autoinstall config,
offline packages, playbook, webapp, and boot files for zero-touch
Proxmox VM deployment
- Add boot-files/ directory for WinPE boot files (wimboot, boot.wim,
BCD, ipxe.efi, etc.) sourced from WestJeff playbook
- Update build-usb.sh and test-vm.sh to bundle boot-files automatically
- Add usb_root variable to playbook, fix all file copy paths to use it
- Unify Apache VirtualHost config (merge default site + webapp proxy)
- Add CSRF token protection to all webapp POST forms and API endpoints
- Update README with Proxmox deployment instructions
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Webapp now listens on port 9009 (UFW rule added)
- Apache reverse proxy updated to proxy to 9009
- test-vm.sh creates a KVM test environment with:
- CIDATA ISO built from project files
- Isolated libvirt network (10.9.100.0/24)
- Ubuntu 24.04 VM with autoinstall
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Samba share at \\server\blancco-reports for automatic report collection
- Webapp reports page with list, download, and delete
- Compliance warning on delete confirmation
- Sidebar link under Tools section
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Added wimtools to offline packages and playbook verification
- Webapp startnet.cmd editor: extract, view, edit, save back to boot.wim
- Uses wimextract/wimupdate for in-place WIM modification
- Dark-themed code editor with tab support and common command reference
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- iPXE boot menu with WinPE, Clonezilla, Blancco Drive Eraser, Memtest86+
- prepare-boot-tools.sh to download/extract boot tool binaries
- Clonezilla backup management in webapp (upload, download, delete)
- Clonezilla Samba share for network backup/restore
- GE Aerospace logo and favicon in webapp
- Updated playbook with boot tool directories and webapp env vars
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Changed from Deploy/Control/unattend.xml to Deploy/FlatUnattendW10.xml
to match the real GE Aerospace image layout.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- webapp/: Flask web management app with:
- Dashboard showing image types and service status
- USB import page for WinPE deployment content
- Unattend.xml visual editor (driver paths, specialize commands,
OOBE settings, first logon commands, raw XML view)
- API endpoints for services and image management
- SETUP.md: Complete setup documentation for streamlined process
- build-usb.sh: Now copies webapp and optional WinPE images to USB
- playbook: Added webapp deployment (systemd service, Apache reverse
proxy), offline package verification, WinPE auto-import from USB
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
