ge-aerospace/pxe-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

webapp: imaging UX overhaul + image management CRUD

Browse Source 
Imaging dashboard
- services/imaging_log_tail.py: parses dnsmasq leases, Apache access log,
  Samba per-host log files, and dnsmasq syslog (DHCP/TFTP). Synthesizes
  inferred sessions keyed by MAC for bays that have only touched the boot
  chain but not yet pushed to /imaging/status. Active window 90 min.
- imaging_status.list_sessions() merges inferred sessions into the dashboard
  list. Real client-pushed sessions win for the same MAC.
- imaging_status: stage_history field tracks every stage transition (capped
  30); sidecar .log file per serial records every log_lines push uncapped
  (read_full_log() caps detail-page response to 1 MB).
- delete_session/delete_all_sessions clean up sidecar .log too.
- New SSE endpoint /imaging/stream emits a session-list hash every 5s.
  Client fetches /imaging/tiles (HTML partial) on hash change and swaps
  #imaging-tiles innerHTML. Polling fallback at 15s if SSE drops.
- Tile-swap preserves scroll, filter input, expanded state via localStorage,
  and any LAPS input the operator is mid-pasting (swap skipped when a
  laps-input is focused).
- imaging.html: removed 15s location.reload(). Added live-status dot in
  header (gray idle / green SSE connected / red SSE lost).
- _imaging_tiles.html: shared partial used by both /imaging full render and
  /imaging/tiles SSE refresh. Inferred bays render with yellow border +
  log-inferred badge + no progress bar (stage_index inference is coarse).
- imaging_detail.html (new): per-bay forensics page at /imaging/session/
  <serial>. Session metadata grid, stage timeline table, full sidecar log
  with truncation indicator, Copy-support-summary button. Linked from each
  client-pushed tile.
- qr-render.js exposes window.renderAllQRs() so the SSE swap can re-render
  Intune device-ID QRs in the swapped-in tiles.

Image management
- services/image_registry.py: JSON registry of image types at
  {SAMBA_SHARE}/image-registry.json. Bootstraps from baked-in
  config.IMAGE_TYPES on first run. create/clone/delete/rename_friendly
  mutate the file then call reload() which rewrites config.IMAGE_TYPES +
  config.FRIENDLY_NAMES in place. Sidebar reflects on next request.
- app.py routes: /images/new, /images/<t>/clone, /images/<t>/delete (with
  optional content-wipe checkbox), /images/<t>/rename.
- dashboard.html: + New image type button + Clone/Delete per row, all in
  Bootstrap modals with confirmation copy.
- Clone copies Deploy/ tree but preserves symlinks to shared dirs (Out-of-
  box Drivers, Operating Systems, Packages) so disk usage stays low.
- Delete with content checked unlinks symlinks (does not follow into shared
  dirs).

Driver / package upload + orphan adoption
- services/images.py: upload_driver, adopt_orphan, remove_orphans,
  upload_package. Filename sanitization blocks path traversal.
- app.py routes: /images/<t>/drivers/upload, /images/<t>/drivers/adopt,
  /images/<t>/drivers/orphans/delete, /images/<t>/packages/upload.
- image_config.html: Upload .zip button + modal on Drivers section. Orphan
  drivers card-footer rebuilt as interactive list with per-row Adopt inline
  form (family + destinationDir inputs) and bulk select+delete.
- Upload .zip on Packages section with optional destinationDir field that
  appends a packages.json entry.

Configuration
- config.py: new env vars DNSMASQ_LEASES, APACHE_ACCESS_LOG, SAMBA_LOG_DIR,
  DNSMASQ_SYSLOG for the log-tailer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
cproudlock
2026-05-30 13:21:06 -04:00
parent c74148a222
commit 69a1682a7f
12 changed files with 2034 additions and 251 deletions
Download Patch File Download Diff File Expand all files Collapse all files

273
webapp/app.py
View File

@@ -12,15 +12,18 @@ This file is the route surface; most logic lives in ``services/``:
services.wim - boot.wim startnet.cmd extract/update via wimtools
"""
import hashlib
import json
import os
import shutil
import tempfile
import time
from datetime import datetime
from pathlib import Path
from flask import (
Flask,
Response,
abort,
flash,
jsonify,
@@ -28,13 +31,24 @@ from flask import (
render_template,
request,
send_file,
stream_with_context,
url_for,
)
from lxml import etree
from werkzeug.utils import secure_filename
import config
from services import blancco_report, deploy, fs, images, imaging_status, system, unattend, wim
from services import (
blancco_report,
deploy,
fs,
image_registry,
images,
imaging_status,
system,
unattend,
wim,
)
from services.audit import audit
from services.csrf import init_csrf
@@ -44,6 +58,10 @@ app.config["MAX_CONTENT_LENGTH"] = config.MAX_CONTENT_LENGTH
init_csrf(app)
# Pull IMAGE_TYPES/FRIENDLY_NAMES from the registry file (created from the
# baked-in config.py defaults on first run). Mutates config.* in place.
image_registry.reload()
# ---------------------------------------------------------------------------
# Routes - pages
@@ -281,6 +299,171 @@ def image_config_save(image_type):
return redirect(url_for("image_config", image_type=image_type))
# ---------------------------------------------------------------------------
# Routes - Driver/package upload + orphan adoption
# ---------------------------------------------------------------------------
@app.route("/images/<image_type>/drivers/upload", methods=["POST"])
def images_drivers_upload(image_type):
if image_type not in config.IMAGE_TYPES:
flash(f"Unknown image type: {image_type}", "danger")
return redirect(url_for("dashboard"))
f = request.files.get("driver_file")
if not f or not f.filename:
flash("No file selected.", "danger")
return redirect(url_for("image_config", image_type=image_type))
family = (request.form.get("family") or "").strip()
dest = (request.form.get("destination_dir") or "").strip()
overwrite = request.form.get("overwrite") == "1"
try:
rec = images.upload_driver(image_type, f, family=family,
destination_dir=dest, overwrite=overwrite)
audit("DRIVER_UPLOAD", f"{image_type}/{rec['filename']} registered={rec['registered']}")
msg = f"Uploaded {rec['filename']}."
msg += " Registered to family in HardwareDriver.json." if rec["registered"] else " Now in orphans list until adopted."
flash(msg, "success")
except (ValueError, FileExistsError, FileNotFoundError) as ex:
flash(str(ex), "danger")
except Exception as ex:
flash(f"Upload failed: {ex}", "danger")
return redirect(url_for("image_config", image_type=image_type))
@app.route("/images/<image_type>/drivers/adopt", methods=["POST"])
def images_drivers_adopt(image_type):
if image_type not in config.IMAGE_TYPES:
flash(f"Unknown image type: {image_type}", "danger")
return redirect(url_for("dashboard"))
filename = (request.form.get("filename") or "").strip()
family = (request.form.get("family") or "").strip()
dest = (request.form.get("destination_dir") or "").strip()
try:
rec = images.adopt_orphan(image_type, filename, family, dest)
audit("DRIVER_ADOPT", f"{image_type}/{rec['filename']} family={family}")
if rec.get("already_registered"):
flash(f"{rec['filename']} was already registered.", "info")
else:
flash(f"Adopted {rec['filename']} into HardwareDriver.json.", "success")
except (ValueError, FileNotFoundError) as ex:
flash(str(ex), "danger")
except Exception as ex:
flash(f"Adopt failed: {ex}", "danger")
return redirect(url_for("image_config", image_type=image_type))
@app.route("/images/<image_type>/drivers/orphans/delete", methods=["POST"])
def images_drivers_orphans_delete(image_type):
if image_type not in config.IMAGE_TYPES:
flash(f"Unknown image type: {image_type}", "danger")
return redirect(url_for("dashboard"))
# Filenames come as repeated form fields (one per checkbox).
filenames = request.form.getlist("filename")
if not filenames:
flash("No files selected for removal.", "warning")
return redirect(url_for("image_config", image_type=image_type))
rec = images.remove_orphans(image_type, filenames)
audit("DRIVER_ORPHAN_REMOVE", f"{image_type} removed={len(rec['removed'])} missing={len(rec['missing'])}")
parts = []
if rec["removed"]:
parts.append(f"Removed {len(rec['removed'])} orphan(s).")
if rec["missing"]:
parts.append(f"{len(rec['missing'])} not found / errored.")
flash(" ".join(parts) or "No-op.", "success" if rec["removed"] else "warning")
return redirect(url_for("image_config", image_type=image_type))
@app.route("/images/<image_type>/packages/upload", methods=["POST"])
def images_packages_upload(image_type):
if image_type not in config.IMAGE_TYPES:
flash(f"Unknown image type: {image_type}", "danger")
return redirect(url_for("dashboard"))
f = request.files.get("package_file")
if not f or not f.filename:
flash("No file selected.", "danger")
return redirect(url_for("image_config", image_type=image_type))
dest = (request.form.get("destination_dir") or "").strip()
overwrite = request.form.get("overwrite") == "1"
try:
rec = images.upload_package(image_type, f, destination_dir=dest, overwrite=overwrite)
audit("PACKAGE_UPLOAD", f"{image_type}/{rec['filename']} registered={rec['registered']}")
msg = f"Uploaded {rec['filename']}."
msg += " Registered in packages.json." if rec["registered"] else " Not registered (no destination_dir provided)."
flash(msg, "success")
except (ValueError, FileExistsError) as ex:
flash(str(ex), "danger")
except Exception as ex:
flash(f"Upload failed: {ex}", "danger")
return redirect(url_for("image_config", image_type=image_type))
# ---------------------------------------------------------------------------
# Routes - Image type CRUD (registry-backed)
# ---------------------------------------------------------------------------
@app.route("/images/new", methods=["POST"])
def images_new():
key = (request.form.get("key") or "").strip().lower()
friendly_name = (request.form.get("friendly_name") or "").strip()
try:
rec = image_registry.create(key, friendly_name)
audit("IMAGE_REGISTRY_CREATE", f"{rec['key']} ({rec['friendly_name']})")
flash(f"Created image type {rec['key']}.", "success")
except image_registry.RegistryError as ex:
flash(str(ex), "danger")
return redirect(url_for("dashboard"))
@app.route("/images/<image_type>/clone", methods=["POST"])
def images_clone(image_type):
if image_type not in config.IMAGE_TYPES:
flash(f"Unknown source image type: {image_type}", "danger")
return redirect(url_for("dashboard"))
dst_key = (request.form.get("dst_key") or "").strip().lower()
friendly_name = (request.form.get("friendly_name") or "").strip() or None
try:
rec = image_registry.clone(image_type, dst_key, friendly_name)
audit("IMAGE_REGISTRY_CLONE", f"{image_type} -> {rec['key']}")
flash(f"Cloned {image_type} -> {rec['key']}.", "success")
except image_registry.RegistryError as ex:
flash(str(ex), "danger")
except Exception as ex:
flash(f"Clone failed: {ex}", "danger")
return redirect(url_for("dashboard"))
@app.route("/images/<image_type>/delete", methods=["POST"])
def images_delete(image_type):
if image_type not in config.IMAGE_TYPES:
flash(f"Unknown image type: {image_type}", "danger")
return redirect(url_for("dashboard"))
delete_content = request.form.get("delete_content") == "1"
try:
rec = image_registry.delete(image_type, delete_content=delete_content)
audit("IMAGE_REGISTRY_DELETE", f"{rec['key']} content={rec['removed_content']}")
msg = f"Removed image type {rec['key']} from registry."
if rec["removed_content"]:
msg += " On-disk content wiped."
flash(msg, "success")
except image_registry.RegistryError as ex:
flash(str(ex), "danger")
return redirect(url_for("dashboard"))
@app.route("/images/<image_type>/rename", methods=["POST"])
def images_rename(image_type):
if image_type not in config.IMAGE_TYPES:
flash(f"Unknown image type: {image_type}", "danger")
return redirect(url_for("dashboard"))
friendly_name = (request.form.get("friendly_name") or "").strip()
try:
rec = image_registry.rename_friendly(image_type, friendly_name)
audit("IMAGE_REGISTRY_RENAME", f"{rec['key']} -> {rec['friendly_name']}")
flash(f"Renamed {rec['key']} to '{rec['friendly_name']}'.", "success")
except image_registry.RegistryError as ex:
flash(str(ex), "danger")
return redirect(url_for("dashboard"))
# ---------------------------------------------------------------------------
# Routes - Clonezilla Backups
# ---------------------------------------------------------------------------
@@ -462,6 +645,75 @@ def imaging_dashboard():
return render_template("imaging.html", sessions=sessions)
@app.route("/imaging/tiles")
def imaging_tiles_partial():
"""HTML fragment of the per-bay tile loop only, used by the dashboard's
SSE/polling refresh to swap #imaging-tiles innerHTML without a full page
reload."""
sessions = imaging_status.list_sessions()
return render_template("_imaging_tiles.html", sessions=sessions)
def _sessions_hash() -> str:
"""Compact fingerprint of the current session list. Used by the SSE
stream to detect changes without sending the full payload. Hashing
(serial-or-key, status, stage_index, current_stage, last_updated) covers
every field the dashboard renders prominently."""
sessions = imaging_status.list_sessions()
h = hashlib.sha256()
for s in sessions:
key = s.get("serial") or f"{s.get('mac','')}-{s.get('ip','')}"
h.update(repr((
key,
s.get("source", "client"),
s.get("status", ""),
s.get("stage_index", 0),
s.get("current_stage", ""),
s.get("last_updated", ""),
s.get("laps_password", "") and "1" or "0",
)).encode())
return h.hexdigest()[:16]
@app.route("/imaging/stream")
def imaging_stream():
"""Server-Sent Events stream of session-list change pings.
Emits one JSON event every SSE_PING_INTERVAL seconds. When the hash
changes from the previously sent value, the client fetches /imaging/tiles
and re-renders. A keepalive heartbeat is sent on the same cadence so
intermediate proxies don't close the connection.
Single-threaded dev server can only serve one SSE client at a time. The
live PXE box runs gunicorn with multiple workers (see playbook) so this
is fine in production.
"""
SSE_PING_INTERVAL = 5 # seconds between hash checks
SSE_MAX_DURATION = 600 # cap connection length so the worker recycles
@stream_with_context
def gen():
start = time.time()
last = None
while time.time() - start < SSE_MAX_DURATION:
try:
cur = _sessions_hash()
except Exception as ex:
yield f": error {ex}\n\n"
cur = last
payload = json.dumps({"hash": cur, "ts": int(time.time())})
yield f"data: {payload}\n\n"
last = cur
time.sleep(SSE_PING_INTERVAL)
headers = {
"Cache-Control": "no-cache",
"X-Accel-Buffering": "no", # disable nginx/apache proxy buffering
"Connection": "keep-alive",
}
return Response(gen(), mimetype="text/event-stream", headers=headers)
@app.route("/imaging/status", methods=["POST"])
def imaging_status_post():
# CSRF-exempt machine-to-machine endpoint; see services/csrf.py exempt list.
@@ -485,6 +737,25 @@ def imaging_session_json(serial):
return jsonify(s)
@app.route("/imaging/session/<serial>")
def imaging_session_detail(serial):
"""Per-bay forensics page: stage timeline, full sidecar log, all session
metadata. Linked from the dashboard tile. Returns 404 if no session JSON
exists for the serial."""
serial = secure_filename(serial)
s = imaging_status.get_session(serial)
if not s:
flash(f"No session for serial {serial}.", "danger")
return redirect(url_for("imaging_dashboard"))
full_log, truncated = imaging_status.read_full_log(serial)
return render_template(
"imaging_detail.html",
session=s,
full_log=full_log,
full_log_truncated=truncated,
)
@app.route("/imaging/delete/<serial>", methods=["POST"])
def imaging_delete_session(serial):
serial = secure_filename(serial)