Commit Graph

5 Commits

Author SHA1 Message Date
cproudlock
bca04f4ca2 Verified MCL uploads: hash every file, repair over HTTP, gate the import
Small Control JSONs have arrived empty or wrong more than once, and the old
check compared sizes only and skipped missing files. A copy of
Deploy/Control on the dev PC still holds two 0-byte JSONs.

Upload-Image.ps1 (tested on Windows PowerShell 5.1 in the win11 VM):
- refuses an unfinished MCL cache: exactly one <OS>-Production.tag (MCL
  writes it last), no Control file newer than it, selected OS matches,
  every Control/Tools JSON non-empty and valid
- checks payload against the SHA-256 MCL's own Control JSONs publish. MCL
  resumes downloads onto old files (FileMode.Append) and then trusts any
  file of the right size: the September junk-tail corruption. Offers to
  delete the bad cache files so MCL downloads them again
- hashes every file into a manifest, robocopies, POSTs the manifest
- re-sends bad small files over HTTP (no proxy), large ones by deleting
  the bad copy and robocopy /J (robocopy skips a same-size, same-time file
  as "modified" even with /IS /IT)

Webapp:
- services/upload_verify.py hashes the upload on the server's own disk in
  a background thread; reports missing, differing, bad-in-MCL-cache and
  stray files; PUT /api/upload/file writes temp, checks hash, renames
- import of the upload folder only after a passed verification, with
  every file unchanged (size, mtime, ctime, inode), no stray files, and
  the target the OS allows (LTSC: Shop Floor only; else Standard or
  Engineer; unknown OS: none). Import, verify and PUT share one lock
- paths matched case-insensitively against the disk, as Samba does
- import page shows the upload, MCL run, verification and stray files
  with a remove button; record of each import kept on the image

Reviewed adversarially; all 11 confirmed findings fixed with tests.
42 new unit tests. VM test: clean upload; same-size same-mtime damage on
3 files repaired in one round; stray file blocks import until removed;
Shop Floor refused for a Pro upload; import records and clears the
folder; missing tag, 0-byte JSON, junk-head WIM and late Control file
each refused with nothing uploaded.
2026-10-16 10:09:40 -04:00
cproudlock
b691a82530 Hardware coverage: flag models the server has no drivers or BIOS for
WinPE now reports each machine's WMI model, manufacturer, Dell system
SKU and BIOS version to /hardware/seen (winpe-status-push.ps1, best
effort, wmic only). The webapp records every model seen and checks it
against what the server has:
  - each image's Deploy/Control/HardwareDriver.json, matched exactly as
    PESetup's GetDriverByModel does (as reimplemented in
    lint-driver-catalogue.py), plus whether the pack's zip is on disk;
  - the WinPE BIOS manifest (substring match like check-bios.cmd).

/hardware lists every model with per-image driver status and BIOS
status. A model missing a Win11 pack or a BIOS entry becomes item 5 on
the Technician To-Do (and the banner count) until the pack/BIOS is added
or it is dismissed. Virtual machines are ignored.

Also deploys the repo winpe-status-push.ps1 over the live copy, which
predated the -CurrentStage/-StageIndex parameters startnet.cmd passes
for the BIOS-flash stage, so that status post was failing.
2026-10-09 09:23:52 -04:00
cproudlock
44a7eeae95 blancco: CMMC evidence pack, ServiceNow tick list, cloud-upload status
Automates the Blancco parts of the onsite-support CMMC playbook:

- ENT-056/158 evidence pack: /reports/evidence?month=YYYY-MM returns
  ENT-056_<Site>_<YYYY-MM>.zip (summary.csv, README, each report's XML
  and PDF). pxe-blancco-evidence.timer saves last month's pack to the
  reports share's evidence/ folder on the 1st (Persistent).
- ENT-045/159 ServiceNow tick list: /reports/servicenow lists recent
  reports as "Wiped" or "Blancco Wipe failed" with a pre-written
  destruction note, copy buttons and Done (CSRF-protected).
- Cloud upload: blancco-progress.py posts each report's upload result
  ("Sending report document id" -> "Report sent to MC") to
  /reports/cloud-status (CSRF-exempt like /imaging/status); the Reports
  page and the CSVs show it, so a wipe missing from BMP is visible.
- CSV gains failure_reason (exception messages, failed steps, error
  counts), business_location and cloud_upload.
- Preferences: business_location "West Jefferson" (config.BLANCCO_SITE)
  and pdf+xml reports; the report copier now copies the PDF too, named
  after its XML. .part and hidden files are not listed as reports.

State (cloud confirmations, ServiceNow ticks) is a JSON file at
/var/lib/pxe-webapp/blancco-state.json, off the report share.
2026-10-05 13:43:16 -04:00
cproudlock
9122b28c31 webapp: imaging progress dashboard + serial column on reports list
Adds end-to-end progress tracking for PXE imaging sessions and surfaces
each Blancco report's BIOS serial in the report list.

webapp:
  * services/imaging_status.py - JSON-per-serial state store under
    IMAGING_DIR (default /var/log/pxe-imaging). Atomic write via
    tempfile + rename. log_tail capped at 50 lines. Merges partial
    updates so clients can post just the current_stage tick.
  * config.py - new IMAGING_DIR env-overridable path.
  * services/csrf.py - explicit exempt list for machine-to-machine
    endpoints; /imaging/status is the first entry. Air-gapped LAN;
    trust-by-network for client posts.
  * app.py - four new routes:
      GET  /imaging               dashboard (renders all sessions)
      POST /imaging/status        client status push (JSON body)
      GET  /imaging/<serial>.json raw session JSON for ad-hoc polling
      POST /imaging/delete/<s>    clear a session from the dashboard
    Also parses each Blancco XML in the /reports list to surface
    system.serial + system.model columns.
  * templates/imaging.html - Bootstrap dashboard with per-session
    cards (state badge, progress bar, stage idx/total, mac, elapsed,
    log tail). meta http-equiv refresh=5 for auto-tick.
  * templates/base.html - new "Imaging Progress" nav entry.
  * templates/reports.html - Serial + Model columns added.

playbook:
  * shopfloor-setup/Shopfloor/lib/Send-PxeStatus.ps1 - new helper.
    Dot-source this then call Send-PxeStatus -Stage X -StageIndex N
    -StageTotal M from any stage script. BIOS serial via CIM, MAC via
    Get-NetAdapter, pctype + machinenumber from C:\Enrollment.
    Failures are swallowed to a local log so a network blip doesn't
    block imaging.
  * shopfloor-setup/Run-ShopfloorSetup.ps1 - dot-sources helper +
    posts at three coarse milestones (start, PPKG enrollment,
    handoff to Monitor-IntuneProgress).
  * shopfloor-setup/gea-shopfloor-keyence/09-Setup-Keyence.ps1 -
    posts at session start + after Install-FromManifest with
    succeeded/failed status derived from $rc. Other 09-Setup-*.ps1
    scripts can follow the same pattern.

ID is BIOS serial (stable across WinPE -> Windows transition and
across reboots, unlike hostname which is random pre-PPKG). Operator
already knows the serial of the bay they imaged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-13 10:07:18 -04:00
cproudlock
c16a4f23b4 webapp: extract service layer (config.py + services/) from app.py
Phase 1a of a multi-session refactor toward a clean blueprint
structure. Pulls the helper code that lived alongside the routes in
the 1621-line app.py into focused modules. app.py is now 625 lines
of mostly routes plus a small Flask wiring header. Behaviour is
unchanged: smoke-tested against the 8 main GET routes (200 OK).

New modules:

- config.py            env vars + IMAGE_TYPES + FRIENDLY_NAMES +
                       SHARED_DEPLOY_* taxonomy + unattend XML
                       namespaces.
- services/audit.py    audit log file handler + audit() helper.
- services/csrf.py     session CSRF token + before_request validator
                       wired via init_csrf(app).
- services/fs.py       image_root / deploy_path / unattend_path /
                       control_path / tools_path + load_json /
                       save_json + resolve_destination.
- services/system.py   service_status / find_usb_mounts /
                       find_upload_sources.
- services/images.py   image_status + load_image_config.
- services/deploy.py   import_deploy + _merge_tree +
                       _replace_with_symlink + allowed_import_source.
- services/unattend.py parse_unattend / build_unattend_xml /
                       extract_form_data and the qn / qwcm / settings
                       pass helpers.
- services/wim.py      extract_startnet / update_startnet / list_files
                       wrapping wimextract / wimupdate / wimdir.

Endpoint names kept stable (dashboard, clonezilla_backups, etc.) so
existing url_for(...) calls in templates are unchanged. Phase 1b
(Flask blueprints with ".endpoint" naming) deferred to a future
session because it requires updating ~30 url_for sites in templates
and is mostly cosmetic.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 18:25:32 -04:00