ge-aerospace/pxe-server
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

webapp: imaging UX overhaul + image management CRUD

Browse Source 
Imaging dashboard
- services/imaging_log_tail.py: parses dnsmasq leases, Apache access log,
  Samba per-host log files, and dnsmasq syslog (DHCP/TFTP). Synthesizes
  inferred sessions keyed by MAC for bays that have only touched the boot
  chain but not yet pushed to /imaging/status. Active window 90 min.
- imaging_status.list_sessions() merges inferred sessions into the dashboard
  list. Real client-pushed sessions win for the same MAC.
- imaging_status: stage_history field tracks every stage transition (capped
  30); sidecar .log file per serial records every log_lines push uncapped
  (read_full_log() caps detail-page response to 1 MB).
- delete_session/delete_all_sessions clean up sidecar .log too.
- New SSE endpoint /imaging/stream emits a session-list hash every 5s.
  Client fetches /imaging/tiles (HTML partial) on hash change and swaps
  #imaging-tiles innerHTML. Polling fallback at 15s if SSE drops.
- Tile-swap preserves scroll, filter input, expanded state via localStorage,
  and any LAPS input the operator is mid-pasting (swap skipped when a
  laps-input is focused).
- imaging.html: removed 15s location.reload(). Added live-status dot in
  header (gray idle / green SSE connected / red SSE lost).
- _imaging_tiles.html: shared partial used by both /imaging full render and
  /imaging/tiles SSE refresh. Inferred bays render with yellow border +
  log-inferred badge + no progress bar (stage_index inference is coarse).
- imaging_detail.html (new): per-bay forensics page at /imaging/session/
  <serial>. Session metadata grid, stage timeline table, full sidecar log
  with truncation indicator, Copy-support-summary button. Linked from each
  client-pushed tile.
- qr-render.js exposes window.renderAllQRs() so the SSE swap can re-render
  Intune device-ID QRs in the swapped-in tiles.

Image management
- services/image_registry.py: JSON registry of image types at
  {SAMBA_SHARE}/image-registry.json. Bootstraps from baked-in
  config.IMAGE_TYPES on first run. create/clone/delete/rename_friendly
  mutate the file then call reload() which rewrites config.IMAGE_TYPES +
  config.FRIENDLY_NAMES in place. Sidebar reflects on next request.
- app.py routes: /images/new, /images/<t>/clone, /images/<t>/delete (with
  optional content-wipe checkbox), /images/<t>/rename.
- dashboard.html: + New image type button + Clone/Delete per row, all in
  Bootstrap modals with confirmation copy.
- Clone copies Deploy/ tree but preserves symlinks to shared dirs (Out-of-
  box Drivers, Operating Systems, Packages) so disk usage stays low.
- Delete with content checked unlinks symlinks (does not follow into shared
  dirs).

Driver / package upload + orphan adoption
- services/images.py: upload_driver, adopt_orphan, remove_orphans,
  upload_package. Filename sanitization blocks path traversal.
- app.py routes: /images/<t>/drivers/upload, /images/<t>/drivers/adopt,
  /images/<t>/drivers/orphans/delete, /images/<t>/packages/upload.
- image_config.html: Upload .zip button + modal on Drivers section. Orphan
  drivers card-footer rebuilt as interactive list with per-row Adopt inline
  form (family + destinationDir inputs) and bulk select+delete.
- Upload .zip on Packages section with optional destinationDir field that
  appends a packages.json entry.

Configuration
- config.py: new env vars DNSMASQ_LEASES, APACHE_ACCESS_LOG, SAMBA_LOG_DIR,
  DNSMASQ_SYSLOG for the log-tailer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
cproudlock
2026-05-30 13:21:06 -04:00
parent c74148a222
commit 69a1682a7f
12 changed files with 2034 additions and 251 deletions
Download Patch File Download Diff File Expand all files Collapse all files

130
webapp/services/imaging_status.py
View File

@@ -20,8 +20,16 @@ from typing import Optional
import config
# How many recent log lines to keep per session.
# How many recent log lines to keep per session in the JSON (dashboard tile
# quick view). The full unbounded log is appended to a sidecar .log file
# next to the .json so the detail page can show everything.
LOG_TAIL_MAX = 50
# Cap how many stage transitions we record per session (bounds JSON size on
# pathological loops; 30 covers more than any real run uses).
STAGE_HISTORY_MAX = 30
# Detail page caps how many bytes of the sidecar .log it sends to the
# browser, to avoid blowing up the response for a runaway log.
DETAIL_LOG_MAX_BYTES = 1024 * 1024 # 1 MB
# Sessions older than this are considered stale and dropped from the dashboard
# "active" list. Still readable individually.
ACTIVE_WINDOW_HOURS = 6
@@ -40,6 +48,30 @@ def _path_for(serial: str) -> str:
return os.path.join(config.IMAGING_DIR, f"{safe}.json")
def _log_path_for(serial: str) -> str:
safe = _SAFE_SERIAL.sub("_", serial.strip()) or "unknown"
return os.path.join(config.IMAGING_DIR, f"{safe}.log")
def _append_full_log(serial: str, lines):
"""Best-effort append to the per-serial sidecar log file. Each line is
timestamped. Failures are swallowed (status-tracking is not the
authoritative log source; the .log is a convenience for the detail
page)."""
if not lines:
return
if isinstance(lines, str):
lines = [lines]
try:
_ensure_dir()
with open(_log_path_for(serial), "a") as f:
ts = _now_iso()
for line in lines:
f.write(f"{ts} {line}\n")
except OSError:
pass
def _now_iso() -> str:
return datetime.now().astimezone().isoformat(timespec="seconds")
@@ -100,7 +132,9 @@ def update_session(payload: dict) -> dict:
# Fresh state after a rewind - mint a new started_at.
state["started_at"] = _now_iso()
# Append any new log lines (preserve old; cap to LOG_TAIL_MAX).
# Append any new log lines: capped tail in the JSON for the dashboard
# quick view, and unbounded append to the sidecar .log for the detail
# page.
new_lines = payload.pop("log_lines", None)
if new_lines:
if isinstance(new_lines, str):
@@ -108,6 +142,33 @@ def update_session(payload: dict) -> dict:
tail = list(state.get("log_tail", []))
tail.extend(new_lines)
state["log_tail"] = tail[-LOG_TAIL_MAX:]
_append_full_log(serial, new_lines)
# Stage history: record a transition row whenever stage_index increases
# or status changes. Bounded to STAGE_HISTORY_MAX so a bouncing client
# can't blow up the JSON. The dashboard tile only needs current state;
# the detail page renders the timeline from this list.
history = list(state.get("stage_history", []))
try:
new_idx = int(payload.get("stage_index") or 0)
except (TypeError, ValueError):
new_idx = 0
try:
old_idx = int(state.get("stage_index") or 0)
except (TypeError, ValueError):
old_idx = 0
new_status = payload.get("status") or state.get("status") or "in_progress"
old_status = state.get("status") or ""
stage_changed = new_idx > old_idx
status_changed = new_status != old_status and new_status in ("succeeded", "failed")
if stage_changed or status_changed or not history:
history.append({
"ts": _now_iso(),
"stage_index": new_idx or old_idx,
"current_stage": payload.get("current_stage") or state.get("current_stage", ""),
"status": new_status,
})
state["stage_history"] = history[-STAGE_HISTORY_MAX:]
for key, value in payload.items():
if value is None or value == "":
@@ -132,8 +193,14 @@ def update_session(payload: dict) -> dict:
return state
def list_sessions() -> list[dict]:
"""Return all sessions sorted by last_updated desc."""
def list_sessions(include_inferred: bool = True) -> list[dict]:
"""Return all sessions sorted by last_updated desc.
When include_inferred is True (default for the dashboard), also pull
server-side log-tail evidence and append synthesized sessions for any
bay that has touched DHCP/TFTP/boot.wim but not yet pushed status.
Real client-pushed sessions always win for the same MAC.
"""
_ensure_dir()
out: list[dict] = []
for name in os.listdir(config.IMAGING_DIR):
@@ -145,7 +212,11 @@ def list_sessions() -> list[dict]:
out.append(json.load(f))
except (json.JSONDecodeError, OSError):
continue
out.sort(key=lambda s: s.get("last_updated", ""), reverse=True)
if include_inferred:
from services import imaging_log_tail
out = imaging_log_tail.merge_with_client_sessions(out)
else:
out.sort(key=lambda s: s.get("last_updated", ""), reverse=True)
return out
@@ -160,27 +231,58 @@ def get_session(serial: str) -> Optional[dict]:
return None
def read_full_log(serial: str, max_bytes: int = DETAIL_LOG_MAX_BYTES) -> tuple[str, bool]:
"""Return (text, truncated). Reads the trailing max_bytes of the sidecar
.log. `truncated` is True when the file was larger than max_bytes and
the leading slice was dropped."""
path = _log_path_for(serial)
try:
size = os.path.getsize(path)
except OSError:
return ("", False)
truncated = size > max_bytes
start = max(0, size - max_bytes)
try:
with open(path, "rb") as f:
f.seek(start)
data = f.read()
except OSError:
return ("", False)
text = data.decode("utf-8", errors="replace")
if truncated:
nl = text.find("\n")
if nl >= 0:
text = text[nl + 1:]
return (text, truncated)
def delete_session(serial: str) -> bool:
path = _path_for(serial)
if not os.path.isfile(path):
return False
try:
os.unlink(path)
return True
except OSError:
return False
# Best-effort sidecar cleanup.
try:
os.unlink(_log_path_for(serial))
except OSError:
pass
return True
def delete_all_sessions() -> int:
"""Wipe every per-bay JSON in IMAGING_DIR. Returns count removed."""
"""Wipe every per-bay JSON + sidecar .log in IMAGING_DIR. Returns count
of JSON files removed."""
_ensure_dir()
removed = 0
for fn in os.listdir(config.IMAGING_DIR):
if not fn.endswith(".json"):
continue
try:
os.unlink(os.path.join(config.IMAGING_DIR, fn))
removed += 1
except OSError:
pass
if fn.endswith(".json") or fn.endswith(".log"):
try:
os.unlink(os.path.join(config.IMAGING_DIR, fn))
except OSError:
continue
if fn.endswith(".json"):
removed += 1
return removed