webapp: imaging UX overhaul + image management CRUD

Imaging dashboard
- services/imaging_log_tail.py: parses dnsmasq leases, Apache access log,
  Samba per-host log files, and dnsmasq syslog (DHCP/TFTP). Synthesizes
  inferred sessions keyed by MAC for bays that have only touched the boot
  chain but not yet pushed to /imaging/status. Active window 90 min.
- imaging_status.list_sessions() merges inferred sessions into the dashboard
  list. Real client-pushed sessions win for the same MAC.
- imaging_status: stage_history field tracks every stage transition (capped
  30); sidecar .log file per serial records every log_lines push uncapped
  (read_full_log() caps detail-page response to 1 MB).
- delete_session/delete_all_sessions clean up sidecar .log too.
- New SSE endpoint /imaging/stream emits a session-list hash every 5s.
  Client fetches /imaging/tiles (HTML partial) on hash change and swaps
  #imaging-tiles innerHTML. Polling fallback at 15s if SSE drops.
- Tile-swap preserves scroll, filter input, expanded state via localStorage,
  and any LAPS input the operator is mid-pasting (swap skipped when a
  laps-input is focused).
- imaging.html: removed 15s location.reload(). Added live-status dot in
  header (gray idle / green SSE connected / red SSE lost).
- _imaging_tiles.html: shared partial used by both /imaging full render and
  /imaging/tiles SSE refresh. Inferred bays render with yellow border +
  log-inferred badge + no progress bar (stage_index inference is coarse).
- imaging_detail.html (new): per-bay forensics page at /imaging/session/
  <serial>. Session metadata grid, stage timeline table, full sidecar log
  with truncation indicator, Copy-support-summary button. Linked from each
  client-pushed tile.
- qr-render.js exposes window.renderAllQRs() so the SSE swap can re-render
  Intune device-ID QRs in the swapped-in tiles.

Image management
- services/image_registry.py: JSON registry of image types at
  {SAMBA_SHARE}/image-registry.json. Bootstraps from baked-in
  config.IMAGE_TYPES on first run. create/clone/delete/rename_friendly
  mutate the file then call reload() which rewrites config.IMAGE_TYPES +
  config.FRIENDLY_NAMES in place. Sidebar reflects on next request.
- app.py routes: /images/new, /images/<t>/clone, /images/<t>/delete (with
  optional content-wipe checkbox), /images/<t>/rename.
- dashboard.html: + New image type button + Clone/Delete per row, all in
  Bootstrap modals with confirmation copy.
- Clone copies Deploy/ tree but preserves symlinks to shared dirs (Out-of-
  box Drivers, Operating Systems, Packages) so disk usage stays low.
- Delete with content checked unlinks symlinks (does not follow into shared
  dirs).

Driver / package upload + orphan adoption
- services/images.py: upload_driver, adopt_orphan, remove_orphans,
  upload_package. Filename sanitization blocks path traversal.
- app.py routes: /images/<t>/drivers/upload, /images/<t>/drivers/adopt,
  /images/<t>/drivers/orphans/delete, /images/<t>/packages/upload.
- image_config.html: Upload .zip button + modal on Drivers section. Orphan
  drivers card-footer rebuilt as interactive list with per-row Adopt inline
  form (family + destinationDir inputs) and bulk select+delete.
- Upload .zip on Packages section with optional destinationDir field that
  appends a packages.json entry.

Configuration
- config.py: new env vars DNSMASQ_LEASES, APACHE_ACCESS_LOG, SAMBA_LOG_DIR,
  DNSMASQ_SYSLOG for the log-tailer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
cproudlock
2026-05-30 13:21:06 -04:00
parent c74148a222
commit 69a1682a7f
12 changed files with 2034 additions and 251 deletions

View File

@@ -2,17 +2,17 @@
{% block title %}Imaging Progress - PXE Server Manager{% endblock %}
{% block extra_head %}
{# JS-driven refresh instead of meta http-equiv so we can cancel it while a #}
{# LAPS-password QR is showing (otherwise the 5s reload wipes the in-page #}
{# state every cycle). #}
{# Tile refresh is driven by SSE (/imaging/stream) with a polling fallback. #}
{# Replacing the full-page reload preserves scroll, filter input, expanded #}
{# tile state, and LAPS QR input text across refreshes. #}
<script>
function scheduleImagingReload() {
window._imagingReloadTimer = setTimeout(function() { location.reload(); }, 15000);
// Polling fallback only; SSE is the primary path. Initialized in
// imaging-refresh.js block at the bottom of the page.
}
function cancelImagingReload() {
if (window._imagingReloadTimer) { clearTimeout(window._imagingReloadTimer); window._imagingReloadTimer = null; }
if (window._imagingPollTimer) { clearTimeout(window._imagingPollTimer); window._imagingPollTimer = null; }
}
window.addEventListener('DOMContentLoaded', scheduleImagingReload);
</script>
{% endblock %}
@@ -21,10 +21,11 @@ window.addEventListener('DOMContentLoaded', scheduleImagingReload);
<div class="d-flex justify-content-between align-items-center mb-2">
<div>
<h2 class="mb-0">Imaging Progress</h2>
<small class="text-muted">Auto-refresh 15s. POST updates from imaging clients arrive at <code>/imaging/status</code>.</small>
<small class="text-muted">Live via SSE (15s polling fallback). Client pushes -> <code>/imaging/status</code>; log-inferred bays in yellow.</small>
</div>
<div class="d-flex align-items-center gap-2">
<span class="badge bg-secondary fs-6"><span id="visible-count">{{ sessions|length }}</span>/{{ sessions|length }}</span>
<span id="imaging-live-dot" class="status-dot" title="live stream" style="background-color:#6c757d;"></span>
<span class="badge bg-secondary fs-6"><span id="visible-count">{{ sessions|length }}</span>/<span id="total-count">{{ sessions|length }}</span></span>
{% if sessions %}
<form method="post" action="{{ url_for('imaging_delete_all') }}"
onsubmit="return confirm('Clear all {{ sessions|length }} imaging session(s)? This wipes every tile from the dashboard. Live re-images will repopulate on next status push.');"
@@ -38,172 +39,24 @@ window.addEventListener('DOMContentLoaded', scheduleImagingReload);
<div class="mb-3">
<input id="imaging-search" type="search" class="form-control form-control-sm"
placeholder="Filter by serial, hostname, pctype, machine#, Intune id, stage name, stage-N, status - typing pauses auto-refresh"
placeholder="Filter by serial, hostname, pctype, machine#, Intune id, MAC, IP, stage name, stage-N, status, source (client|inferred)"
autocomplete="off">
</div>
{% if not sessions %}
<div class="card">
<div id="imaging-empty" class="card">
<div class="card-body text-center text-muted py-5">
<p class="mb-1">No imaging sessions yet.</p>
<p class="small mb-0">A PC being imaged will post status here.
Until then, nothing to show.</p>
<p class="small mb-0">A PC being imaged will post status here, or appear
automatically once it touches DHCP / TFTP / boot.wim.</p>
</div>
</div>
{% endif %}
{% set stage_labels = {
1: ('Booting from PXE', 'WinPE loaded - applying Windows image to disk.'),
2: ('Configuring Windows', 'First boot. Running shopfloor setup baseline scripts.'),
3: ('Installing apps', 'Type-specific app installs (eDNC, UDC, NTLARS, etc).'),
4: ('Apps installed', 'Type-specific scripts complete. Preparing for Intune enrollment.'),
5: ('Enrolling in Intune', 'PPKG installing - device joining Azure AD + Intune. ~5-10 min, reboot to follow.'),
6: ('Waiting on first Intune sync','Post-PPKG settle (~120s). Triggering Schedule #3 sync repeatedly.'),
7: ('Registered - assign category',
'Phase 1 (Intune Registration) complete. Click "set category" in the Intune portal to drop the bay into the right config-profile group.'),
8: ('Imaging complete',
'Lockdown applied. Bay rebooted into ShopFloor session. Ready for production.')
} %}
<div id="imaging-tiles">
{% include "_imaging_tiles.html" %}
</div>
{% for s in sessions %}
{% set stage_idx = s.stage_index | int(0) %}
{% set stage_total = s.stage_total | int(0) %}
{% set pct = 100 if s.status == 'succeeded' else ((stage_idx / stage_total * 100) | round(0, 'floor')) if stage_total > 0 else 0 %}
{% set is_failed = s.status == 'failed' %}
{% set is_done = s.status == 'succeeded' %}
{% set border = 'danger' if is_failed else ('success' if is_done else 'primary') %}
{% set friendly = stage_labels.get(stage_idx, ('Stage ' ~ stage_idx, '')) %}
{# Stage 1 sub-phase: if WinPE pushed a BIOS update stage string, #}
{# show a BIOS-specific friendly label. Otherwise default idx=1. #}
{% if stage_idx == 1 and s.current_stage and 'bios' in s.current_stage|lower %}
{% set friendly = ('Updating BIOS firmware',
'WinPE detected a firmware update for this model. Do NOT power off until the next reboot. Imaging continues afterward.') %}
{% endif %}
{# Stage 7 fans out by sub-phase. Monitor pushes different stage #}
{# strings as it crosses each Phase 1-4 boundary. Swap friendly #}
{# label based on which keyword shows up. #}
{% if stage_idx == 7 and s.current_stage %}
{% set _cs = s.current_stage|lower %}
{% if 'ready for lockdown' in _cs or 'request lockdown' in _cs %}
{% set friendly = ('Ready - request lockdown',
'Phase 1-4 all complete (Registration, Device Config, Software Deploy, Credentials). Click "ARTS request" to initiate the lockdown workflow.') %}
{% elif 'credentials' in _cs or 'phase 4' in _cs %}
{% set friendly = ('Phase 3 / 4 - DSC + credentials',
'SFLD policy delivered, DSC pulling device-config.yaml + running per-app wrappers. SFLD share creds populating.') %}
{% elif 'sfld policy' in _cs or 'phase 2' in _cs or 'device configuration' in _cs %}
{% set friendly = ('Phase 2 - device configuration',
'Category was assigned in Intune. SFLD ConfigurationProfile (Function + SasToken) has delivered. DSC kicking off next.') %}
{% endif %}
{% endif %}
<details class="card border-{{ border }} mb-2 shadow-sm imaging-card"
data-serial="{{ s.serial }}"
data-filter="{{ s.serial|lower }} {{ (s.hostname_target or '')|lower }} {{ (s.pctype or '')|lower }} {{ (s.machinenumber or '')|lower }} {{ (s.intune_device_id or '')|lower }} {{ friendly[0]|lower }} stage-{{ stage_idx }} {{ (s.status or 'in_progress')|lower }}">
<summary class="card-body py-2" style="cursor:pointer; list-style:none;">
<div class="d-flex flex-wrap gap-3 align-items-center">
{% if s.intune_device_id %}
<div data-qr="{{ s.intune_device_id }}" data-qr-size="96" data-qr-ec="M"
style="line-height:0; flex-shrink:0;"
title="Intune Device ID: {{ s.intune_device_id }}"></div>
{% else %}
<div class="d-flex align-items-center justify-content-center bg-light text-muted small"
style="width:96px; height:96px; border-radius:0.25rem; flex-shrink:0; text-align:center; padding:0.25rem;">
no DeviceId
</div>
{% endif %}
<div class="flex-grow-1" style="min-width:0;">
<div class="d-flex flex-wrap align-items-center gap-2">
<strong class="fs-6">{{ s.serial or '(no serial)' }}</strong>
{% if s.hostname_target %}<code class="text-muted small">{{ s.hostname_target }}</code>{% endif %}
{% if s.pctype %}<span class="badge bg-info text-dark">{{ s.pctype }}</span>{% endif %}
{% if s.machinenumber %}<span class="badge bg-secondary">#{{ s.machinenumber }}</span>{% endif %}
<span class="badge bg-{{ border }} ms-auto">{{ s.status or 'in_progress' }}</span>
</div>
<div class="d-flex justify-content-between align-items-baseline mt-1">
<div>
<strong>{{ friendly[0] }}</strong>
<span class="badge bg-secondary ms-1">{{ stage_idx }}/{{ stage_total or '?' }}</span>
</div>
<span class="text-muted small">{{ pct }}%</span>
</div>
<div class="progress mt-1" style="height:0.7rem;">
<div class="progress-bar bg-{{ border }} {% if not is_done and not is_failed %}progress-bar-striped progress-bar-animated{% endif %}"
role="progressbar" style="width: {{ pct }}%;"
aria-valuenow="{{ pct }}" aria-valuemin="0" aria-valuemax="100"></div>
</div>
</div>
</div>
</summary>
<div class="card-body pt-0 pb-3 border-top">
{% if friendly[1] %}<div class="small text-muted mt-2">{{ friendly[1] }}</div>{% endif %}
{% if s.intune_device_id %}
<div class="small mt-2" style="font-size:0.75rem;">
<span class="text-muted">Intune:</span> <code>{{ s.intune_device_id }}</code>
<button type="button" class="btn btn-sm btn-outline-secondary py-0 px-1 copy-btn"
style="font-size:0.65rem; line-height:1; transition: all 0.2s;"
data-copy-text="{{ s.intune_device_id }}">copy</button>
<a class="btn btn-sm btn-outline-primary py-0 px-1"
style="font-size:0.65rem; line-height:1;"
target="_blank" rel="noopener"
href="https://portal.azure.us/?feature.msaljs=false#view/Microsoft_Intune_Devices/DeviceSettingsMenuBlade/~/properties/aadDeviceId/{{ s.intune_device_id }}">set category</a>
<a class="btn btn-sm btn-outline-warning py-0 px-1"
style="font-size:0.65rem; line-height:1;"
target="_blank" rel="noopener"
href="https://arts.dw.geaerospace.net/requests/type">ARTS request</a>
</div>
{% endif %}
<div class="text-muted mt-1" style="font-size:0.7rem;">
<span class="me-3">started <code>{{ s.started_at or '-' }}</code></span>
<span class="me-3">last <code>{{ s.last_updated or '-' }}</code></span>
<span class="me-3">MAC <code>{{ s.mac or '-' }}</code></span>
{% if s.current_stage %}<span style="font-family:monospace;">{{ s.current_stage }}</span>{% endif %}
</div>
{% if s.error %}
<div class="alert alert-danger small py-2 mb-2 mt-3">
<strong>Error:</strong> {{ s.error }}
</div>
{% endif %}
<div class="mt-3 laps-card" data-serial="{{ s.serial }}">
<div class="text-muted small mb-1">LAPS password QR (paste -> scan on bay - persists until cleared)</div>
<div class="d-flex align-items-center gap-2">
<input type="text"
class="form-control form-control-sm laps-input"
style="font-family: monospace; max-width: 22rem;"
placeholder="paste LAPS password from Intune portal here"
autocomplete="off"
value="{{ s.laps_password or '' }}">
<button type="button" class="btn btn-sm btn-primary laps-make-btn">{% if s.laps_password %}Update QR{% else %}Make QR{% endif %}</button>
<button type="button" class="btn btn-sm btn-outline-secondary laps-clear-btn" {% if not s.laps_password %}style="display:none;"{% endif %}>Clear</button>
</div>
<div class="laps-qr-container mt-2"></div>
</div>
{% if s.log_tail %}
<details class="mt-3">
<summary class="text-muted small">Log tail ({{ s.log_tail | length }} line{{ 's' if s.log_tail | length != 1 }})</summary>
<pre class="bg-light p-2 mt-2 small mb-0" style="max-height: 12rem; overflow-y: auto;">{% for line in s.log_tail %}{{ line }}
{% endfor %}</pre>
</details>
{% endif %}
<div class="mt-3 text-end">
<form method="post" action="{{ url_for('imaging_delete_session', serial=s.serial) }}" style="display: inline;">
<input type="hidden" name="_csrf_token" value="{{ csrf_token() }}">
<button type="submit" class="btn btn-sm btn-outline-secondary"
onclick="return confirm('Clear session {{ s.serial }}?');">
Clear
</button>
</form>
</div>
</div>
</details>
{% endfor %}
<div class="card mt-3">
<div class="card-body small text-muted">
@@ -230,6 +83,104 @@ Content-Type: application/json
{% block extra_scripts %}
<script>
// -------- Live refresh: SSE primary, polling fallback --------
// Rebuilds the #imaging-tiles inner HTML from /imaging/tiles when the
// server signals a state change. Preserves scroll, filter input value,
// and any LAPS input that the operator is actively editing.
(function() {
var TILES_URL = "{{ url_for('imaging_tiles_partial') }}";
var STREAM_URL = "{{ url_for('imaging_stream') }}";
var POLL_MS = 15000;
var lastHash = null;
var sse = null;
var dot = function() { return document.getElementById('imaging-live-dot'); };
function setDot(color, title) {
var d = dot();
if (d) { d.style.backgroundColor = color; d.title = title || ''; }
}
function lapsInputIsDirty() {
// Skip the tile swap if any LAPS input is focused (operator is
// mid-paste) OR has unsaved text that differs from the server-side
// copy. The next refresh after they hit Make-QR will catch up.
var active = document.activeElement;
if (active && active.classList && active.classList.contains('laps-input')) return true;
return false;
}
function refreshTiles(force) {
if (!force && lapsInputIsDirty()) return;
fetch(TILES_URL, { credentials: 'same-origin' })
.then(function(r) { return r.text(); })
.then(function(html) {
var container = document.getElementById('imaging-tiles');
if (!container) return;
container.innerHTML = html;
if (typeof window.imagingPostSwapHooks === 'function') {
window.imagingPostSwapHooks();
}
})
.catch(function(err) { console.error('refreshTiles failed:', err); });
}
function startPolling() {
if (window._imagingPollTimer) return;
window._imagingPollTimer = setInterval(function() {
refreshTiles(false);
}, POLL_MS);
}
function stopPolling() {
if (window._imagingPollTimer) {
clearInterval(window._imagingPollTimer);
window._imagingPollTimer = null;
}
}
function startSSE() {
if (!window.EventSource) {
setDot('#ffc107', 'EventSource unsupported - polling only');
startPolling();
return;
}
try {
sse = new EventSource(STREAM_URL);
} catch (e) {
setDot('#dc3545', 'SSE failed - polling');
startPolling();
return;
}
sse.onopen = function() {
setDot('#198754', 'live stream connected');
stopPolling();
};
sse.onmessage = function(ev) {
var data;
try { data = JSON.parse(ev.data); } catch (e) { return; }
if (!data || data.hash === lastHash) return;
lastHash = data.hash;
refreshTiles(false);
};
sse.onerror = function() {
setDot('#dc3545', 'live stream lost - polling fallback');
try { sse.close(); } catch (e) {}
sse = null;
startPolling();
// Try to reconnect SSE after a backoff.
setTimeout(startSSE, 10000);
};
}
// Expose so external code (LAPS, filter) can trigger an immediate
// refresh after user action.
window.imagingRefreshNow = function() { refreshTiles(true); };
window.addEventListener('DOMContentLoaded', function() {
startSSE();
});
})();
function copyText(text) {
// Modern path - only works over HTTPS or localhost
if (navigator.clipboard && window.isSecureContext) {
@@ -362,67 +313,87 @@ document.addEventListener('keydown', function(e) {
if (card) { e.preventDefault(); renderLapsQR(card); }
}
});
// On page load, any laps-card with a pre-populated input (from
// server-persisted laps_password) auto-renders its QR without re-POSTing.
window.addEventListener('DOMContentLoaded', function() {
document.querySelectorAll('.laps-card').forEach(function(card) {
var input = card.querySelector('.laps-input');
if (input && input.value) renderLapsQR(card, { skipPersist: true });
});
});
// Persist tile expanded/collapsed state across page refresh via
// localStorage. Set of expanded serials lives at 'imaging-expanded'.
// Per-tile hooks that must re-run after every tile-swap. Called on
// DOMContentLoaded for first paint, then by the SSE/polling refresh after
// it replaces the innerHTML of #imaging-tiles.
(function() {
var KEY = 'imaging-expanded';
function loadSet() {
try { return new Set(JSON.parse(localStorage.getItem(KEY) || '[]')); }
var EXPANDED_KEY = 'imaging-expanded';
function loadExpandedSet() {
try { return new Set(JSON.parse(localStorage.getItem(EXPANDED_KEY) || '[]')); }
catch (e) { return new Set(); }
}
function saveSet(set) {
try { localStorage.setItem(KEY, JSON.stringify(Array.from(set))); }
function saveExpandedSet(set) {
try { localStorage.setItem(EXPANDED_KEY, JSON.stringify(Array.from(set))); }
catch (e) {}
}
window.addEventListener('DOMContentLoaded', function() {
var expanded = loadSet();
document.querySelectorAll('.imaging-card').forEach(function(card) {
var serial = card.getAttribute('data-serial');
if (serial && expanded.has(serial)) card.open = true;
card.addEventListener('toggle', function() {
var s = loadSet();
if (card.open) s.add(serial); else s.delete(serial);
saveSet(s);
});
});
});
})();
// Client-side filter: hide imaging-card elements whose data-filter doesn't
// match the search query. Live as user types. Pauses auto-reload while
// the input is focused or non-empty so typing isn't interrupted by refresh.
window.addEventListener('DOMContentLoaded', function() {
var search = document.getElementById('imaging-search');
var counter = document.getElementById('visible-count');
if (!search) return;
function applyFilter() {
var q = search.value.trim().toLowerCase();
var visible = 0;
function restoreExpandedState() {
var expanded = loadExpandedSet();
document.querySelectorAll('.imaging-card').forEach(function(card) {
var serial = card.getAttribute('data-serial') || card.getAttribute('data-key');
if (serial && expanded.has(serial)) card.open = true;
if (!card._toggleBound) {
card.addEventListener('toggle', function() {
var s = loadExpandedSet();
if (card.open) s.add(serial); else s.delete(serial);
saveExpandedSet(s);
});
card._toggleBound = true;
}
});
}
function autoRenderLapsQRs() {
document.querySelectorAll('.laps-card').forEach(function(card) {
var input = card.querySelector('.laps-input');
var container = card.querySelector('.laps-qr-container');
if (input && input.value && container && !container.innerHTML.trim()) {
renderLapsQR(card, { skipPersist: true });
}
});
}
function renderIntuneQRs() {
// qr-render.js looks for [data-qr] and renders an image. It runs on
// initial DOMContentLoaded but not after a tile-swap. Re-run if the
// hook is exposed; otherwise no-op.
if (typeof window.renderAllQRs === 'function') window.renderAllQRs();
}
function applyFilter() {
var search = document.getElementById('imaging-search');
var counter = document.getElementById('visible-count');
var total = document.getElementById('total-count');
if (!search) return;
var q = search.value.trim().toLowerCase();
var visible = 0, totalN = 0;
document.querySelectorAll('.imaging-card').forEach(function(card) {
totalN++;
var hay = card.getAttribute('data-filter') || '';
var match = (q === '') || hay.indexOf(q) !== -1;
card.style.display = match ? '' : 'none';
if (match) visible++;
});
if (counter) counter.textContent = visible;
if (q !== '') { cancelImagingReload(); }
else { cancelImagingReload(); scheduleImagingReload(); }
if (total) total.textContent = totalN;
}
search.addEventListener('input', applyFilter);
search.addEventListener('focus', cancelImagingReload);
search.addEventListener('blur', function() {
if (search.value.trim() === '') { cancelImagingReload(); scheduleImagingReload(); }
window.imagingPostSwapHooks = function() {
restoreExpandedState();
autoRenderLapsQRs();
renderIntuneQRs();
applyFilter();
};
window.addEventListener('DOMContentLoaded', function() {
// Search input is rendered outside #imaging-tiles, so its listeners
// only bind once.
var search = document.getElementById('imaging-search');
if (search) {
search.addEventListener('input', applyFilter);
}
window.imagingPostSwapHooks();
});
applyFilter();
});
})();
</script>
{% endblock %}